-- Logs begin at Mon 2022-09-12 11:15:12 UTC, end at Mon 2022-09-12 12:15:25 UTC. -- Sep 12 11:38:38.628799 dv-cinder-853315-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 12 11:38:38.635976 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 12 11:38:38 2022] *** Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: nodename: dv-cinder-853315-9-iscsi Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: machine: x86_64 Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: clock source: unix Sep 12 11:38:38.637977 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: pcre jit disabled Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: detected number of CPU cores: 4 Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: current working directory: / Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: detected binary path: /usr/bin/uwsgi-core Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: your processes number limit is 47680 Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: your memory page size is 4096 bytes Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: detected max file descriptor number: 2048 Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: lock engine: pthread robust mutexes Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: thunder lock: enabled Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 12 11:38:38.638425 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 12 11:38:38.678061 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: Python main interpreter initialized at 0x5602e257d9c0 Sep 12 11:38:38.678061 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: python threads support enabled Sep 12 11:38:38.678061 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: your server socket listen backlog is limited to 100 connections Sep 12 11:38:38.678061 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: your mercy for graceful operations on workers is 90 seconds Sep 12 11:38:38.678461 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: mapped 403077 bytes (393 KB) for 2 cores Sep 12 11:38:38.678585 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: *** Operational MODE: preforking *** Sep 12 11:38:38.678800 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: *** uWSGI is running in multiple interpreter mode *** Sep 12 11:38:38.678800 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: spawned uWSGI master process (pid: 81377) Sep 12 11:38:38.678992 dv-cinder-853315-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 12 11:38:38.681415 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: spawned uWSGI worker 1 (pid: 81386, cores: 1) Sep 12 11:38:38.681415 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: spawned uWSGI worker 2 (pid: 81387, cores: 1) Sep 12 11:38:38.681415 dv-cinder-853315-9-iscsi devstack@keystone.service[81377]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 12 11:38:42.965113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 11:38:42.965113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: from cryptography.utils import int_from_bytes Sep 12 11:38:42.965938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 11:38:42.965938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: from cryptography.utils import int_from_bytes Sep 12 11:38:42.968178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 11:38:42.968178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: from cryptography.utils import int_from_bytes Sep 12 11:38:42.968633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 11:38:42.968633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: from cryptography.utils import int_from_bytes Sep 12 11:38:43.015673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 11:38:43.015861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 11:38:43.016054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] command line args: [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 11:38:43.016323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 11:38:43.016502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ================================================================================ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 11:38:43.016755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] admin_token = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.017002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_dir = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.017190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.017273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 11:38:43.017373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] config_source = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.017455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 11:38:43.017553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] debug = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.017641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] command line args: [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 11:38:43.017753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 11:38:43.018070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ================================================================================ {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 11:38:43.018134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] admin_token = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] config_dir = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.018941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] config_source = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] debug = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_config_append = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_dir = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.019957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_config_append = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.020988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_dir = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_file = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.021894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.022973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.023904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] publish_errors = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.024825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] standard_threads = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] publish_errors = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.025964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_journal = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_json = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_stderr = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] use_syslog = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026788 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] standard_threads = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.026967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.027088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.027200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.027311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.027420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.027535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.027656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.027803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] use_journal = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.027803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] use_json = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.028019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.028130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] use_stderr = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.028246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.028365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] use_syslog = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.028483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.028594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 11:38:43.028704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.028819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.028929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.029864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.030535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.030663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.030783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.030894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.031906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.032928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.033909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.034963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.035904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.036900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.037922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.038941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.039965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.040957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.041979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.042917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.043939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.044985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.045940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.046981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.047895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.048919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.049894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.050931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.external = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.051969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.password = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.052981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] auth.token = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.053826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.external = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.054976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.password = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.055925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] auth.token = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.056946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.057878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.058905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.059951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.060985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.061959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.062990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.063969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.064889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.065963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.066900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.067914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.068975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.069832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.070062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.070301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.070554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.070801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.071885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.072958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.073895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.074927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.075887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.076574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.077909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.078983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.079910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.080973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.081927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.082999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.083973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.084915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.085978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.086956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.087980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.088976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.089924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.090903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.091989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.092109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.092224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.092353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] role.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] role.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.093888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.094923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.095856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.096919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.097946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.098944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.099966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.100998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.101906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.102988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.103100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 11:38:43.103212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.103368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.103616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.103874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.104114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.104357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.104598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.104848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.105136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.105371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.105618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.105887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.106152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.connection = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.106400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.106675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.106919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.107166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.107405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.107645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.107885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.108268 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.108727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.109045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.109300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.109544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.109799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.110041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.110303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.110558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.110808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.111048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 11:38:43.111223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81386) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 11:38:43.132294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.132508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.132741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.137263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.137501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.137733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.142299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.147154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.171001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.171280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.174387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.174661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.225880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.226386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.274696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.274930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.276175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.276417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.431619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.431810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.431892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.432917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.457998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.458132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.458214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.458350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.458449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.469827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.470398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.482581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.482768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.483993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.484098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.484330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.484557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.484794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81386) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 11:38:43.501465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.501675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 11:38:43.502021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.502227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.502309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 11:38:43.502442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 11:38:43.502543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.502781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.502861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.503012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 11:38:43.503106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.503291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.503374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.503556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.503820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.504086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.512928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.513212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.513485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.513752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.513832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.514042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.514930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.515447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.521695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.522505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.524635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.524926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.525213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.525449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.525530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.525740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.525831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.526956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.527981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.528253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.550280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.551779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.551864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.552072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.553276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.553570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.556780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.557069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.557351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.558282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.558596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.558888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.564295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.564586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.566088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.566384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.570213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.570531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.570826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.571116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.571397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.571675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.571950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.572106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.572223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.572409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.572706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.573002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.573289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.573575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.573857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.574136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.588354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.588637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.590973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.591258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.592820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.595455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.596101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.596408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.598743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.599047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.600659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.603339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.603923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.606568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.607193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.607496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.607786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.608296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.608573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.608871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.610219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.610545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.610842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.611131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.611416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.611703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.625177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.625463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.625735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.626002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.628416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.628720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.628997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.629268 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.632885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.634451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.636013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.636205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.636288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.637754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.639058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.639909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.640018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.640200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.640451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.640539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.640860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.643178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.643471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.643759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.644059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.644461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.652057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.652349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.652635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.652920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.653305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.653598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.653884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.656152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.656449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.656739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.657031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.657419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.657709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.657994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.669351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.672810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.675910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.676472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.677424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.678935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.679226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.679488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.679625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.682499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.683985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.685454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.685744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.686015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.686835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.688070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.691002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.691402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.691694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.691979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.693518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.695024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.697928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.698333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.698659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.698942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.699827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.700124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.700418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.707220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.707520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.707602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.707807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.707902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.708124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.708421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.708686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.708962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.709253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.709546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.709822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.710098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.710372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.710673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.710953 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.711242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.716451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81386) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 11:38:43.716743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.717032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.717360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.717823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.718326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.718891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.719340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.719631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.719925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.720222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.720616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.720898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.721180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81386) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 11:38:43.746300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5602e257d9c0 pid: 81387 (default app) Sep 12 11:38:43.750086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a741a6-eaa3-404d-bf77-31f3fdf3adfa None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:43.750373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a741a6-eaa3-404d-bf77-31f3fdf3adfa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:43.750683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a741a6-eaa3-404d-bf77-31f3fdf3adfa None None] PATH_INFO: `/v3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:43.754826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Sep 12 11:38:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:38:43.757348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5602e257d9c0 pid: 81386 (default app) Sep 12 11:38:50.561415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:50.561914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:50.562019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:50.563272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 2/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:38:50.571001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:50.571402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:50.571536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:50.642633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: INFO dbcounter [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] Registered counter for database keystone Sep 12 11:38:50.661772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG oslo_db.sqlalchemy.engines [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81386) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 11:38:50.668143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writer thread running {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 11:38:50.854808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG passlib.handlers.bcrypt [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81386) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 11:38:50.855193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG passlib.handlers.bcrypt [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81386) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 11:38:50.912701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:38:50.914293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:38:51.041239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:38:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 11:38:51.565348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:51.634769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: INFO dbcounter [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] Registered counter for database keystone Sep 12 11:38:51.656721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG oslo_db.sqlalchemy.engines [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81387) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 11:38:51.662817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writer thread running {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 11:38:51.784223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c86b2f-7768-4c9f-9bb6-8155e1a2c0c4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:51.788559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:51.789183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:51.789468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:51.789746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:51.798186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:51.849852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 11:38:51.850202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f984c-b7b1-4be0-8c9a-7564b7b0987f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:51.859814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 3/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Sep 12 11:38:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:38:51.865672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:51.881070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a500f-a623-4a56-ace9-f2ca844a4868 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:51.885451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:51.886591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:51.886873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:51.887142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] PATH_INFO: `/v3/projects/1375001928324c1f9d2a4c3ccf24704e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:51.888584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] RBAC: Authorizing `identity:get_project(project_id=1375001928324c1f9d2a4c3ccf24704e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:51.941663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 11:38:51.941999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb92dfe-b07b-4ec8-aac9-d0d3e03ae638 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:51.946443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 2/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Sep 12 11:38:51 2022] GET /identity/v3/projects/1375001928324c1f9d2a4c3ccf24704e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:38:54.095822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d81a6-02f2-4fc9-beb0-cf2a2bdc9cf2 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:54.096330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d81a6-02f2-4fc9-beb0-cf2a2bdc9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:54.096438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d81a6-02f2-4fc9-beb0-cf2a2bdc9cf2 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:54.100462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 4/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:38:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:38:54.105761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:54.106139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:54.106309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:54.127273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:38:54.128807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:38:54.136668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 3/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:38:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 11:38:54.400545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d81a6-02f2-4fc9-beb0-cf2a2bdc9cf2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:54.412719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d81a6-02f2-4fc9-beb0-cf2a2bdc9cf2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:54.416889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:54.417483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:54.417762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:54.418035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:54.426224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:54.428993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5efa0-692d-4549-84f1-5006978efaf3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:54.437783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 5/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Sep 12 11:38:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 11:38:54.443120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:54.452594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d77a-30ab-4630-9598-4eb64f95a336 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:54.456897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:54.457518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:54.457797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:54.458070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:54.459395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:54.461219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eefe63-381f-48a6-9e14-90c6c3b21ce9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:54.462746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 4/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Sep 12 11:38:54 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:38:56.689476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce45cd-6714-4307-a558-e782a2483385 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:56.690015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce45cd-6714-4307-a558-e782a2483385 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:56.690015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce45cd-6714-4307-a558-e782a2483385 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:56.691365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 6/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:38:56.696648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:56.697068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:56.697323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:56.716004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:38:56.717791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:38:56.725158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d107a8-bbc3-43f0-a27b-c90d9ddf2a93 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:56.725526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d107a8-bbc3-43f0-a27b-c90d9ddf2a93 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:56.725805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d107a8-bbc3-43f0-a27b-c90d9ddf2a93 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:56.727163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 7/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:38:56.732292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:56.732577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:56.732855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:56.774451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG passlib.handlers.bcrypt [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81387) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 11:38:56.774895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG passlib.handlers.bcrypt [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81387) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 11:38:56.799403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 5/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:38:56.824566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:38:56.826740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:38:56.927385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 8/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:38:57.066412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.090624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.173975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c18fc-159b-4ab7-abf7-753a0a0b2350 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.206489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.207027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.207146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.207389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.207657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.273570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.275093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.298686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.307873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04af1f-8a80-4965-9c5e-534e8d81b527 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.310135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.347646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.348754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.349039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.349311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.360871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:38:57.369330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:38:57.370294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:38:57.371715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 9/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:38:57.376571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.377186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 6/15] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:38:57.382448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.476410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc98243-035e-4b9e-8763-ddeb5841b4e6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.478762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03b64-beab-4527-a594-f40f84928ebe None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.500568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.509153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.509759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.510043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.510316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.511010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:38:57.511316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.512454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.512747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.513080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.513358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.514180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.524373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.524757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 10/16] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:38:57.525843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.531089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:38:57.544441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:38:57.545933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:38:57.547098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:38:57.547098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:38:57.547098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:38:57.547098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 7/17] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:38:57.551159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.559670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.633352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672c47a-4563-40b5-af00-0d239902d590 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.649499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879441e1-d6ae-45c4-a45f-22b3ac63726e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.666614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.667255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.667536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.667808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.669308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.671556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.673348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 11/18] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:38:57.678742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.683505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.684110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.684389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.684657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.685297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:38:57.685774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.687188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.704719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 8/19] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:38:57.775888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.784139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abfceb-7c36-48cc-a859-f5a601953c0c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.819334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.819981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.820258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.820529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.822108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.823606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a2346-991b-4f08-aeb6-9d510c8c47e1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.825229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 12/20] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:38:57.831559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:38:57.867838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:38:57.935722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5a7a6-8e68-4648-9cdf-f433229f81d5 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:38:57.971982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:38:57.972592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:38:57.972877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:38:57.973148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:38:57.973884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.domain.id=default, user.id=d0a0e6dd3f1445518b190cea2b73f55b) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:38:57.974195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:38:57.975791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999de6-e4a9-4899-bf7d-cb496f277550 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:38:57.983620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 9/21] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Sep 12 11:38:57 2022] GET /identity/v3/role_assignments?user.id=d0a0e6dd3f1445518b190cea2b73f55b&scope.domain.id=default&role.id=ada491db6331406099f2a486b26a3cc4 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:00.351703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db499d4f-8afe-4a63-883f-f8e6004fa385 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.352219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db499d4f-8afe-4a63-883f-f8e6004fa385 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.352293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db499d4f-8afe-4a63-883f-f8e6004fa385 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.353829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 13/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:00.360576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92380-7d86-4df1-a9a0-52630017ea03 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.361538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92380-7d86-4df1-a9a0-52630017ea03 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.362013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c92380-7d86-4df1-a9a0-52630017ea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.387229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a2c92380-7d86-4df1-a9a0-52630017ea03 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:00.389317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a2c92380-7d86-4df1-a9a0-52630017ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:00.481344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 10/23] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:00.534296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e5e6-4eca-4b2f-9e72-4d63921fd312 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.534647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e5e6-4eca-4b2f-9e72-4d63921fd312 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.534882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e5e6-4eca-4b2f-9e72-4d63921fd312 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.536397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 14/24] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:00.542276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebee-3c71-4964-8c5a-3a3ff324de5a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.542912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebee-3c71-4964-8c5a-3a3ff324de5a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.543381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebee-3c71-4964-8c5a-3a3ff324de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.571857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-170cebee-3c71-4964-8c5a-3a3ff324de5a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:00.573444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-170cebee-3c71-4964-8c5a-3a3ff324de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:00.604147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:00.662886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315c81-8457-4bcd-a10e-f2cb8e7afb00 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.663550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315c81-8457-4bcd-a10e-f2cb8e7afb00 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.664018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f315c81-8457-4bcd-a10e-f2cb8e7afb00 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.665640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 15/25] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:00.671614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.672221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.672720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.683015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 11/26] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:00.704259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:00.705762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:00.769400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608575-2cd6-4d08-8d4a-9e68bc8fb847 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.770046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608575-2cd6-4d08-8d4a-9e68bc8fb847 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.770555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608575-2cd6-4d08-8d4a-9e68bc8fb847 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.772323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 12/27] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:00.789663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 16/28] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:00.793760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:00.794284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:00.795028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:00.823918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:00.825163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:00.849767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:00.937415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 13/29] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:01.005087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:01.018198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845289-35cf-42f6-a823-f89a01b56a04 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.067802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.068451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.068774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.069057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.097274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.102006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:01.103526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:39:01.134049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:01.134996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:39:01.135835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:39:01.135835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:39:01.135835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:01.135835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 17/30] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:01.142440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.215029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:01.231044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff57636-e91f-4437-9b56-084fc30601d7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.286029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.286806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.287093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.288951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.307630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:01.314182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:01.315064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:01.315814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:01.318795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a2f8b-b5d1-4e4f-ace2-d2255ed67866 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.320009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 14/31] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:01.320657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.361758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.362438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.366546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.366546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.366546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:01.366546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:01.366546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:01.397990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 18/32] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:01.399019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.407289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:01.482623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f7a43-0085-46aa-a756-f6293a635591 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.516111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4280acb-803b-4629-acbe-d6b4674ee4b4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.529550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.530351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.530689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.530967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.531908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:01.534038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:01.557534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.558629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.559210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.559796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.561049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:01.561560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:01.563286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:01.573074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 19/33] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:01.574317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.652594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:01.690833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fff2ed-246d-4e1c-89c6-4312f1b3ee39 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.691212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 15/34] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 12 11:39:01 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:01.692644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.721810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:01.737022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.738130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.738422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.738733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.739429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:01.740816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:01.880266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 20/35] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 12 11:39:01 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:01.886581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:01.937661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f946720-97c7-4eb8-9265-68ff69351607 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:01.995811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:01.996793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:01.997281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:01.997826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:01.998739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.000673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.046573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 16/36] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:39:01 2022] POST /identity/v3/projects => generated 301 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:02.079606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c79482-ce9b-4db5-b29e-5a4f8484c26f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:02.119779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:02.131103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:02.131785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.132129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.132413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.135499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.137181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84875fd-2b24-47fa-9fe1-e44d0342bec1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.139324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 21/37] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:01 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:02.145387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:02.222951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:02.268829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848188f0-b1e5-472a-ad6a-d20f3376ccdc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.269066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848188f0-b1e5-472a-ad6a-d20f3376ccdc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.269324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848188f0-b1e5-472a-ad6a-d20f3376ccdc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.271062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 22/38] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:02.272868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05494a43-0d25-41b7-afeb-2a1cf043db59 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:02.276693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.276984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.277264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.308096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:02.311059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:02.340880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:02.342056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.342665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.343150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.344566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.346242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6df80-634e-4c75-961b-288fbac84820 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.348069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 17/39] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:02.349503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06232-4960-4b4c-8f36-340481fdde6d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.350001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06232-4960-4b4c-8f36-340481fdde6d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.350282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06232-4960-4b4c-8f36-340481fdde6d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.351852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 18/40] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:02.357566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.357704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.358105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.379210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:02.380753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:02.401218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 23/41] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:02.402180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:02.521944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8f9bd-4bed-4878-8d14-4cab1f5edbe7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:02.548364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:02.560303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:02.561048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.561358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.561637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] PATH_INFO: `/v3/domains/default/users/d0a0e6dd3f1445518b190cea2b73f55b/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.563764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d0a0e6dd3f1445518b190cea2b73f55b, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.565211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.567815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 19/42] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:02.584889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 24/43] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Sep 12 11:39:02 2022] PUT /identity/v3/domains/default/users/d0a0e6dd3f1445518b190cea2b73f55b/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:02.676045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:02.738201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:02.785035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f25e8-33bd-4554-8a5a-ce50db858e1f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:02.819304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:02.819952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.820239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.820517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.821191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.822639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.835698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:02.841219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 20/44] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 12 11:39:02 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:02.952686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08533c1-b8ed-47f1-b3a9-7c63d30a7d20 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:02.989453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:02.990110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:02.990389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:02.990688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:02.991992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:02.993480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e7ce8-00a2-4827-925d-c499240eb461 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:02.995413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 25/45] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:03.001605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:03.115662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de9240-bf74-4d0e-95ef-0fbbd2fe15b7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:03.133362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:03.151805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:03.152418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:03.152699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:03.152973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:03.153632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:03.155823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa9f63-c044-4302-9672-c75a2c954320 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:03.181040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 21/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:39:03 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:04.757023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ea59e-bcb2-4dfe-bc1f-5e8df2dde5c6 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:04.757569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ea59e-bcb2-4dfe-bc1f-5e8df2dde5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:04.757569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ea59e-bcb2-4dfe-bc1f-5e8df2dde5c6 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:04.759093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 26/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:04.764704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:04.765117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:04.765250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:04.783449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:04.784625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:04.849651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 22/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:05.116345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ea59e-bcb2-4dfe-bc1f-5e8df2dde5c6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.170592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:05.220739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ea59e-bcb2-4dfe-bc1f-5e8df2dde5c6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.257719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.258358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.258678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.258957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.269427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-afd7f24c-3ab4-4b6a-bc34-8c92eb8e91c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:05.274292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:05.275341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:05.276274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 27/49] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:05.283799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.333916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36816-dc97-4ed6-8ad4-918370eea792 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.334200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36816-dc97-4ed6-8ad4-918370eea792 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.334477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36816-dc97-4ed6-8ad4-918370eea792 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.336277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 28/50] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:05.341951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.342248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.342554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.360477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:05.361662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:05.378988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:05.381618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863246ca-2484-465d-a408-78c88d28cf24 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.416699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.417316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.417603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.417879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.418576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:05.418889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:05.420821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:05.429466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 29/51] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:05.432550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 23/52] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:05.438688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.480777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:05.538019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f335-4858-494c-b2c5-5494ee25f871 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.571513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.572140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.572424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.572702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.573844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:05.575326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:05.577492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 30/53] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:05.582920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.684453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6551b1-af5b-4a27-9c7f-640cdd4cc1ba None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.701305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.719890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.720531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.720817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.721096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.722218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:05.723724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:05.725435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 24/54] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:05.730997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.761542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:05.804029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ab2b9-c42f-46fa-b224-de7154b0a618 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.816134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:05.832458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3b041-05ec-4248-ad6b-4ea8a5dc9893 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:05.842660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.843303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.843585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.843869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.844989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:05.846424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:05.848002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 31/55] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:05.853369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:05.865369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:05.865978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:05.866257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:05.866547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:05.867279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.domain.id=default, user.id=d0a0e6dd3f1445518b190cea2b73f55b) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:05.867587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:05.869009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ee54c-562f-4542-9e1c-263e25566355 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:05.875141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 25/56] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Sep 12 11:39:05 2022] GET /identity/v3/role_assignments?user.id=d0a0e6dd3f1445518b190cea2b73f55b&scope.domain.id=default&role.id=ada491db6331406099f2a486b26a3cc4 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:06.065223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a79fd8-f356-4a37-be50-d2f52287c5dd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:06.102269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:06.102915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:06.103200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:06.103479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:06.104164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:06.105606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8122d-bed6-4431-a3c8-ea6c65e3237f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:06.137324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 32/57] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Sep 12 11:39:05 2022] POST /identity/v3/projects => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:39:08.276896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1a064-97c0-414f-b435-fb51a33518ba None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:08.277418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1a064-97c0-414f-b435-fb51a33518ba None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:08.277418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1a064-97c0-414f-b435-fb51a33518ba None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:08.279338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 26/58] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:08.284764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:08.285183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:08.285324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:08.304148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:08.305334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:08.328862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:08.373891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 33/59] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:08.643493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1a064-97c0-414f-b435-fb51a33518ba None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:08.735295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:08.742943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1a064-97c0-414f-b435-fb51a33518ba None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:08.775061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:08.775679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:08.775959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:08.776230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:08.777329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:08.778941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2082c-954a-4a75-bc5d-b4c502f79e45 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:08.780468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 27/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:08.785868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:08.883091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d54f6d-1a2a-4059-9f98-e6adef09ae1b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:08.916502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:08.917068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:08.917349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:08.917623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:08.918284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:08.919891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ab06-e26a-46ac-878c-28a5dc811d96 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:08.945881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 34/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:08 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:11.990097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca9ff-80c1-4fe4-80b9-9d1d4ac873c4 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:11.991131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca9ff-80c1-4fe4-80b9-9d1d4ac873c4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:11.991622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dca9ff-80c1-4fe4-80b9-9d1d4ac873c4 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:11.993213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 28/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:11.999490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.000297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.000778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.022311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:12.023880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:12.040128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:12.120257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 35/63] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:12.531060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ee5-e818-4874-9aa9-bf9d860c6f2e None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.531748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ee5-e818-4874-9aa9-bf9d860c6f2e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.532257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7ee5-e818-4874-9aa9-bf9d860c6f2e None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.533833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 29/64] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:12.537421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:12.540292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e991-9443-4b68-a3c9-1f41506c3f19 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.540848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e991-9443-4b68-a3c9-1f41506c3f19 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.541330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e991-9443-4b68-a3c9-1f41506c3f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.576029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-75a5e991-9443-4b68-a3c9-1f41506c3f19 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:12.578796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-75a5e991-9443-4b68-a3c9-1f41506c3f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:12.683677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430bd8-bc55-4848-b045-715b36935582 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:12.728804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:12.729471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 30/65] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:12.729736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.730027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.730305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.731784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37ae-88cc-4a2c-8f06-6d1fde75f2af None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.732296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37ae-88cc-4a2c-8f06-6d1fde75f2af None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.732760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37ae-88cc-4a2c-8f06-6d1fde75f2af None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.734484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 31/66] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:12.739987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba064-2a9d-443e-87cc-6cf02b9c1818 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.740489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba064-2a9d-443e-87cc-6cf02b9c1818 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.740974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ba064-2a9d-443e-87cc-6cf02b9c1818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.742111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:12.747786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:12.748631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:12.749387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 36/67] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:12.758534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:12.770540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:12.779446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-751ba064-2a9d-443e-87cc-6cf02b9c1818 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:12.782593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-751ba064-2a9d-443e-87cc-6cf02b9c1818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:12.875422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 32/68] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:12.876745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26100eb8-0e22-48f8-834b-901642e54ddd None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.877040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26100eb8-0e22-48f8-834b-901642e54ddd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.877317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26100eb8-0e22-48f8-834b-901642e54ddd None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.882624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 33/69] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:12.884141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6511d21-2f25-4308-9503-68f5749f528d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.884438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6511d21-2f25-4308-9503-68f5749f528d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.884718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6511d21-2f25-4308-9503-68f5749f528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.893997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873c27-f6c3-47b5-96bb-4b7734ede0cc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:12.903608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e6511d21-2f25-4308-9503-68f5749f528d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:12.904868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e6511d21-2f25-4308-9503-68f5749f528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:12.923206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:12.933310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:12.934363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:12.935192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:12.935795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:12.936846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:12.937442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:12.939188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:12.948691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 37/70] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:12.964608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:12.976537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 34/71] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:13.018983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40fc2a-9651-4f09-bc88-56be313055b4 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.020007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40fc2a-9651-4f09-bc88-56be313055b4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.020481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40fc2a-9651-4f09-bc88-56be313055b4 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.022303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 35/72] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:13.035843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.036413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.036899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.074800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:13.077018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:13.077724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.118929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3301d4-751d-4776-acd5-be60db82250f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.154216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.155136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.155557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.155969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.162734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 36/73] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:13.169621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.172129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.173738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.176256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 38/74] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:12 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:13.184078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.286015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.300517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75cb17-72c9-4413-925c-92d9de075ad9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.306218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d1d99-a1ea-4623-a39d-c04a6b9d3493 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.328394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.338236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.338922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.339452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.339731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.347785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.348832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.349118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.349394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.361512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:13.366851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:13.367706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:13.368482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 37/75] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:13.369129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.417823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.420085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.478273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eef27-b731-404b-9fd4-708a7be6ff08 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.511506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.512333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.513265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.513522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.518759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 39/76] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:13.519601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.527568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:13.533925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:13.534852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 38/77] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:13.536659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.540513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.548869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.620617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3309ff-3530-4d0e-b54b-b013d6fe86ea None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.638923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7b37-7c56-4fb1-909d-353538a7e8f1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.659714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.660643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.661105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.661521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.663189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.665479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.673037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 40/78] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:13.673037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.673866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.674479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.674789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.675064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.675738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:13.676037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.677436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.692748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 39/79] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:13.692748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.751887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.764534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.771575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e28475-9808-4245-9a0a-67e121cfcd7c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.789083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b544657-e785-4951-b108-9ac996cc2bd3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.804678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.805305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.805586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.805862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.816326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:13.821139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:13.822032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:13.827466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 41/80] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:13.828096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.828180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.828246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 40/81] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:13.828986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.925486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c944e-62bc-4722-9440-65a6ceda4383 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.926875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03df4-8ddc-432d-93cc-704456ca3790 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:13.954762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:13.959157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.959767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.959989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:13.960098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.960319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.960589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:13.960875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:13.961021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:13.961160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:13.961297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.961792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:13.962716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.963271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:13.977089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 42/82] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:13.977089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:13.994557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 41/83] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:13 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:13.995404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.005759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.104082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd91a9-47ab-4ecc-8298-a1f5ef24bb7b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.129250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb57d-16b8-4c7c-855c-58df98eef40c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.147490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.148258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.148548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.148828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.150168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.151679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.153370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 43/84] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:14.153370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.172143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.172777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.173234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.173512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.174205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:14.174619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.175943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.185766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:14.187722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.246504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.261237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.281956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3707-271b-4366-b340-7f07335664ef None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.327952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.328881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.329171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.329454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.330197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:14.330536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.331824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdc17-f100-46c2-adae-0bf82db52c59 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.332017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.342635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 44/86] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=7c78b3797aea4e3280d32765c45d23df => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:14.343700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.389219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.390169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.390734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.391208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.392784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.395504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.398397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 43/87] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:14.400151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.502957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef9694-b143-4444-8418-15d25a888076 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.519113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848764-4fa2-44c7-b4f7-fb33faa6611f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.539008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.555121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.556125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.556498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.556933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.560968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.561821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.562309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.562832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.564218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.566137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.568119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:14.569355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.581070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.651218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.653059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.716275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c43e-0a14-487d-b3ad-0605422ed7eb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:14.790703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:14.790703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:14.790703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:14.790703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:14.838847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:14.840462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:14.874615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.894495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:14.896044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:14.924274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:14.981296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057286c5-9926-41bb-8373-f1ccd063556f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.024751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.025554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.025944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.026326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.111527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.115503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.135960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 45/90] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:15.136955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.174034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:15.259052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:15.260319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 46/91] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:15.261372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.277275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598221d-0d59-4e1e-83c2-cad1a30a4888 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.320174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.321175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.321716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.322213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.323625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.325440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.327868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:15.329026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.445269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c271930-be87-4d93-9366-95ffd5248a55 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.486090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.487020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.487439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.487835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] PATH_INFO: `/v3/projects/5ffc1609ca45470cad1de533f23407af` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.491146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] RBAC: Authorizing `identity:get_project(project_id=5ffc1609ca45470cad1de533f23407af)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.493350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.497418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/projects/5ffc1609ca45470cad1de533f23407af => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:15.500318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ecc25-c794-4a2a-9f37-6ca5e47d1dcf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.502655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.546223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:15.572515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.578255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.578725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.579187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.580662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.582289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.584315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 47/94] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:15.586894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.618868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:15.694072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a365-654f-4b6c-a0c1-11537ea3279f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.760086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.760936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.761367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.761868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.763018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=d0a0e6dd3f1445518b190cea2b73f55b) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:15.763616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.765869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.776629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 48/95] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/role_assignments?user.id=d0a0e6dd3f1445518b190cea2b73f55b&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:15.779567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062d581-cc12-4399-9280-6eb600ef376d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:15.780681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:15.867501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:15.868384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:15.868906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:15.869367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:15.870410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=5ffc1609ca45470cad1de533f23407af, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:15.870954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:15.873776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc0586-89e4-4276-bd31-9fdc0f7c4893 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:15.880606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=5ffc1609ca45470cad1de533f23407af&role.id=7c78b3797aea4e3280d32765c45d23df => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:15.962978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:15.987138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c7d99-d1fb-4dd7-aba9-83c5d358cde1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:16.044860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:16.045497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:16.045783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:16.046064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:16.046886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:16.047202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:16.048727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175a249-f537-487c-97c0-39574679ca38 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:16.058829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:15 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:16.725062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d53275-98ec-4767-ac5d-0d52d04c4bcd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:16.726150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d53275-98ec-4767-ac5d-0d52d04c4bcd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:16.726676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d53275-98ec-4767-ac5d-0d52d04c4bcd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:16.728500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:16.748552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f68d1-edae-492b-a931-dfa7f429f727 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:16.749044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f68d1-edae-492b-a931-dfa7f429f727 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:16.749145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f68d1-edae-492b-a931-dfa7f429f727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:16.779740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f94f68d1-edae-492b-a931-dfa7f429f727 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:16.780985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f94f68d1-edae-492b-a931-dfa7f429f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:16.875978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 50/99] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:17.125588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de558-50dd-423a-b3a5-c8d0f1a5f6d3 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.126159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de558-50dd-423a-b3a5-c8d0f1a5f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.126666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de558-50dd-423a-b3a5-c8d0f1a5f6d3 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.128489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 50/100] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:17.133562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fee-8cc0-4fbe-a0f8-cef659b008f6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.134100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fee-8cc0-4fbe-a0f8-cef659b008f6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.134616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fee-8cc0-4fbe-a0f8-cef659b008f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.163573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9e4c2fee-8cc0-4fbe-a0f8-cef659b008f6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:17.164766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9e4c2fee-8cc0-4fbe-a0f8-cef659b008f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:17.240865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 51/101] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:17.448166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de558-50dd-423a-b3a5-c8d0f1a5f6d3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:17.519282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f47627-f6e7-4ca4-a721-4241be3902e8 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.519594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f47627-f6e7-4ca4-a721-4241be3902e8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.519831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f47627-f6e7-4ca4-a721-4241be3902e8 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.521176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 52/102] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:17.526398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.526645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.526915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.551344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:17.564194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:17.565411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:17.573394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de558-50dd-423a-b3a5-c8d0f1a5f6d3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:17.573910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:17.626109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:17.627155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.627702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.628280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.644347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:17.649582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:17.650433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:17.651230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:17.654201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 51/103] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:17.659711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:17.702651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 53/104] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:17.795233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5e87-ff53-448d-8093-575c35937f7b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:17.839915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:17.841527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:17.842234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:17.842745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:17.843630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:17.844139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:17.845716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:17.859844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 52/105] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:17.865675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:17.915219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:17.919817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.009519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f213f7d-66a4-43a6-9a6b-a1024621385d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.057678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cf973-ddf0-438d-aef8-fdf5b3d7c246 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.063077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.063717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.063998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.064274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] PATH_INFO: `/v3/users/b4316462724642ef93229e65d3c3c5ce` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.091705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.095419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] RBAC: Authorizing `identity:get_user(user_id=b4316462724642ef93229e65d3c3c5ce)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.096918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.102132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 54/106] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:17 2022] GET /identity/v3/users/b4316462724642ef93229e65d3c3c5ce => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:18.102132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.115243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.116167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.116730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.117208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.120496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.123289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.127444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 53/107] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:18.128677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.235580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.286560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd8b7c-b157-498e-9eea-56585e103b05 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.314309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f41a0-a2a4-4293-9b53-6bc8aa51ca4c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.333845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.334737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.335278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.335704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.365246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.367055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.368240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.369896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.393413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:18.398814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:18.399893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:18.400871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:18.403223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 55/108] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:18.404456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.463582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.465739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.508979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.568560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.596749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aed18e-526a-4522-a9a1-5ca8d17550e1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.636540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 54/109] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:18.640688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.696879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.697912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.698398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.698895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.699752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.702444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.734535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 56/110] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:39:18 2022] POST /identity/v3/projects => generated 302 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:18.735971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.849594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864c8f5-5e53-4c64-9435-83a0be5908f4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.866783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c268-79ee-483f-86da-88512102a729 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:18.881254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:18.912589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.914371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.916382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.916969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.919177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.922198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68164b92-e2b6-4745-9360-baaf92d568b4 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.925345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 57/111] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:18 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:18.928538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4328a-3b36-467b-91dc-643851d2ec7f None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.929971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4328a-3b36-467b-91dc-643851d2ec7f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.931389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4328a-3b36-467b-91dc-643851d2ec7f None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.936300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 58/112] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:18.938348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4328a-3b36-467b-91dc-643851d2ec7f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:18.939316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:18.939965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.940328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.940618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.941347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:18.941657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:18.943621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05166456-f112-4941-a673-fb7b71763c21 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:18.956106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 55/113] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:18.962199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4056-7df5-4522-89f8-7ae63fcc36c7 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.962199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4056-7df5-4522-89f8-7ae63fcc36c7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.962494 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a4056-7df5-4522-89f8-7ae63fcc36c7 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.964169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 56/114] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:18.967379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:18.967669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:18.968008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:18.991446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:18.992675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:19.022238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.071552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4328a-3b36-467b-91dc-643851d2ec7f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:19.079845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 57/115] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:19.080968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.136725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:19.137411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.137833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.138327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.139177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=b4316462724642ef93229e65d3c3c5ce) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:19.139499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:19.143282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df99953-2e07-4659-9e78-0ee72c9a5269 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:19.154546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 59/116] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:18 2022] GET /identity/v3/role_assignments?user.id=b4316462724642ef93229e65d3c3c5ce&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:19.155822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.156114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.156394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.196983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:19.198449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:19.273943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.308527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2ec44-0a4c-4fae-b6aa-c90f501f3d84 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:19.340032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 60/117] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:19.383928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:19.384520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.384862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.385148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.387152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:19.389667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b50d5-09e3-4adf-9d49-6f6cdf5beffa None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:19.391779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 58/118] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:19 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:19.397207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.409320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7bad-ba8b-42cd-9bbd-263df0459132 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.409634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7bad-ba8b-42cd-9bbd-263df0459132 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.409885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf7bad-ba8b-42cd-9bbd-263df0459132 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.411386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 59/119] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:19.416917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.417205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.417490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.437682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:19.439084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:19.485837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.523787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854021-6c69-49ac-9495-4af9480e2702 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:19.546041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 60/120] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:19.546970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.574364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:19.577150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.577657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.578128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.580311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:19.582763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:19.584780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 61/121] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:19 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:19.595441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.652908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.718285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0fb1-7b4b-4d45-97f5-cc8b35d0ac9a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:19.744276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2a9bb-154b-4678-85a3-7a6bf53a0c09 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:19.759243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:19.760278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.760758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.761225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.763998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, user_id=27c6e1813c56499ab5b07dc305f8bdc3, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:19.765695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:19.785247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 62/122] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:19 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:19.786375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.818978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:19.820003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:19.820680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:19.822559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:19.847423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:19.855703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:19.857841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:19.858665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:19.863634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 61/123] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:19.870747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:19.897650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.991033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:19.994895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c931ea-f708-4c56-b7d1-6cde5417f3a2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.070075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.070780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.071070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.071349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.072820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb15e2-3b26-4537-a449-6f9cebeb8e5d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.095829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:20.103093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:20.103947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:20.104716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 63/124] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:20.105345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.130134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.131195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.131675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.132243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.133111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:20.133630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.135423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.148486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 62/125] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.149626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.292024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b841a1-9c15-4c91-88bb-62b1b257060e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.294571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:20.327595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c56cc-68c4-4457-8498-7b8716688262 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.332772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.333654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.334134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.334678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.347775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:20.353173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:20.354023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:20.354839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:20.358081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:20.359194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.384011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:20.404932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.405974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.405974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.406141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.406833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:20.407147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.408553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.430929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 63/127] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.431964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.474537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90da7d-756b-4fa7-b307-7610e8c0c9da None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.510223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.511431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.511915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.512946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.514322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.516024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.518235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 65/128] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.519309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.614920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:20.628293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb04f6-2f93-46f1-a2e3-af746d6df6d2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.673885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.675297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.679653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.680244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.681851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.686138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.691649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 66/129] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.692676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.727770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0ab61-c9ff-455a-aa2c-b5e2fe84fcb3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.768880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.770199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.770752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.771489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.772357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:20.772974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.774890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.785612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 64/130] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.786812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:20.797258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:20.906442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56c4af-a275-462a-801e-c7e941888756 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.945835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a6e1-849c-4ea5-8d1c-9a7a3c90b260 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:20.946654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:20.947701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:20.948174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:20.948642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] PATH_INFO: `/v3/projects/5ffc1609ca45470cad1de533f23407af` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:20.950158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] RBAC: Authorizing `identity:get_project(project_id=5ffc1609ca45470cad1de533f23407af)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:20.952687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:20.954579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 65/131] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/projects/5ffc1609ca45470cad1de533f23407af => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:20.955608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:21.044239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:21.045779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.046442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.046752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.047032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.048189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.049658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.060251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 67/132] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:21.061060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:21.076190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcec3e6-e899-421c-ac47-92d16fc45bfc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:21.130184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.131126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.131628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.132077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.133426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.135181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:21.136625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.139299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 66/133] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:20 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:21.140293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:21.254314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4214d03-ada1-4e6f-bead-4beccc17de90 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:21.292529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.294008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.294900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.295355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/d0a0e6dd3f1445518b190cea2b73f55b/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.297493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, user_id=d0a0e6dd3f1445518b190cea2b73f55b, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.299138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.316449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 67/134] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:21 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/d0a0e6dd3f1445518b190cea2b73f55b/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:21.317565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:21.324057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:21.372932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432fd8-95aa-4f61-82b1-bec4d1bcc160 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:21.468608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd4bb-178c-42d4-a207-85cb7cd2963f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:21.490539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.490539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.490539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.490539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] PATH_INFO: `/v3/projects/5ffc1609ca45470cad1de533f23407af/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.499509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] RBAC: Authorizing `identity:create_grant(project_id=5ffc1609ca45470cad1de533f23407af, user_id=27c6e1813c56499ab5b07dc305f8bdc3, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.500962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dacc1c-436f-4e5f-acee-58e94b34f1ff None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.534070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.534735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.535082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.535365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.537442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.539037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.541187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 68/135] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:21 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:21.544091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 68/136] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:21 2022] PUT /identity/v3/projects/5ffc1609ca45470cad1de533f23407af/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:21.547997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:21.671616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:21.722853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26f81-f3bd-4fa2-ab3c-ef4e7ace1448 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:21.766538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:21.767438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.767730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.768012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.770960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, user_id=27c6e1813c56499ab5b07dc305f8bdc3, role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:21.772470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415ab46-c54c-4ea6-8020-2b0e7aa0e2ab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:21.789936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 69/137] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:21 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/27c6e1813c56499ab5b07dc305f8bdc3/roles/b42a9d0451fc419ab1547c88f1248317 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:39:21.822600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd3a1-929e-4ef6-aed5-88adb3fd8fc2 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.822952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd3a1-929e-4ef6-aed5-88adb3fd8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.823177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd3a1-929e-4ef6-aed5-88adb3fd8fc2 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.824825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 69/138] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:21.831123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799c3f-b0e7-43b7-8e12-61ad3bae1a23 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:21.831418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799c3f-b0e7-43b7-8e12-61ad3bae1a23 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:21.831691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799c3f-b0e7-43b7-8e12-61ad3bae1a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:21.852525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-15799c3f-b0e7-43b7-8e12-61ad3bae1a23 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:21.853759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-15799c3f-b0e7-43b7-8e12-61ad3bae1a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:21.960339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 70/139] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:22.138374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c677-7596-437e-914a-95663d3025b0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:22.138954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c677-7596-437e-914a-95663d3025b0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:22.138954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c677-7596-437e-914a-95663d3025b0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:22.140460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 70/140] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:22.151954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:22.152245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:22.152526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:22.183688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:22.184997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:22.256765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:22.270466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 71/141] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:22.328124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948c677-7596-437e-914a-95663d3025b0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:22.432411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:22.451783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948c677-7596-437e-914a-95663d3025b0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:22.495046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:22.496092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:22.496619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:22.497099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:22.502393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:22.504099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:22.505966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 71/142] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:22.511520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:22.623013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03238f86-527f-4ed1-ac08-4b7bceb17078 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:22.659877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:22.660861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:22.661346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:22.661807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:22.662744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:22.664459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b931-f04c-48a5-a1a0-6adb7c2f8142 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:22.690951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 72/143] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:22 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:22.899985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.029869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1500e-9d22-42c3-b852-2fecb4675aaf None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.052995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701eec7-ba98-4f12-a6ee-022ff9f548a8 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.053949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701eec7-ba98-4f12-a6ee-022ff9f548a8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.054430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701eec7-ba98-4f12-a6ee-022ff9f548a8 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.056229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 73/144] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:23.061754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.062259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.062749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.090980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.091925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.092381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.092835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.094642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:23.097152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:23.117944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:23.126801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:23.127699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:23.128560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 72/145] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:23.133832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.166876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:23.186607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:23.218307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 74/146] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:23.313914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50fe5-0598-4973-85cf-49114d5e5b93 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.364102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.365083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.365564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.366032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.366941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:23.367439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:23.369023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:23.382047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 73/147] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:23.387931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.515976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.527897 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129d7a2-2c43-45f2-850f-ed53390d6b68 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.567628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.568192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.568469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.568738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] PATH_INFO: `/v3/users/b4316462724642ef93229e65d3c3c5ce` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.569864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] RBAC: Authorizing `identity:get_user(user_id=b4316462724642ef93229e65d3c3c5ce)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:23.573483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:23.576244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 75/148] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:23 2022] GET /identity/v3/users/b4316462724642ef93229e65d3c3c5ce => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:23.582094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.641769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:23.709126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a67d3b-0962-4a1e-aec6-3e6ed2534f65 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.710858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:23.749242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.749888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.750742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.751018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.752334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:23.753773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:23.755587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 76/149] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:23 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:23.761852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.784347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3a8f6-18af-4a0e-9514-c5226323896c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.869985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.870690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.870970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.871240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.877085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696659de-18a6-42a1-98b7-f2a9e9cfdc1f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:23.886974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:23.891419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:23.897214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:23.898237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:23.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 74/150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:23.909427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:23.918304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:23.919368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:23.919821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:23.920285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/b4316462724642ef93229e65d3c3c5ce/roles/0b8995b146864249aeeb7d42c6f75d7e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:23.922439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, user_id=b4316462724642ef93229e65d3c3c5ce, role_id=0b8995b146864249aeeb7d42c6f75d7e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:23.924160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:23.941187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 77/151] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:23 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/users/b4316462724642ef93229e65d3c3c5ce/roles/0b8995b146864249aeeb7d42c6f75d7e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:39:24.046970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764690bb-5e9c-4bbd-b63b-65cff93e6b20 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:24.089209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:24.089843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.090666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.090967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.091745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:24.092052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:24.093464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:24.099682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:24.103023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 75/152] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:24.109406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:24.254524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e36f5-1f57-4472-b104-a3751cb2c5c8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:24.293377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:24.294402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.294948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.295410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.296829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:24.298459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626dae11-a483-4dcf-a0fd-5d44e739f780 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:24.300259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 78/153] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:24 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:24.305924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:24.434025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f389c7-4dc3-4160-8ee4-a2d191b6f2ff None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:24.472247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:24.472870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.473150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.473426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.518573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1219e3-ee72-42ab-b7da-4933f0649722 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.519185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1219e3-ee72-42ab-b7da-4933f0649722 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.519649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1219e3-ee72-42ab-b7da-4933f0649722 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.521283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 79/154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:24.528942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1d38-429d-4384-b082-783812d48157 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.529615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1d38-429d-4384-b082-783812d48157 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.530124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e1d38-429d-4384-b082-783812d48157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.541900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:24.549341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:24.550884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:24.553806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e22e1d38-429d-4384-b082-783812d48157 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:24.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e22e1d38-429d-4384-b082-783812d48157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:24.599569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:24.638479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 80/155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:24.659803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 76/156] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:24.665487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e1d38-429d-4384-b082-783812d48157 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:24.771290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e1d38-429d-4384-b082-783812d48157 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:24.807530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:24.808658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.809838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.810318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.811708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:24.813356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:24.815536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 81/157] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:24 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:24.821064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:24.946855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:24.953343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5f0ad-0d47-4617-b5d5-39acdf6d91a3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:24.989777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:24.990413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:24.990727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:24.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:24.991763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:24.992074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:24.993586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72354dba-df74-4c11-9299-cbc8f5955403 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:25.000842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 77/158] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:24 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=7c78b3797aea4e3280d32765c45d23df => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:25.191973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:25.237449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:25.249178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29893cf-4aa4-4f33-98c8-d8d9e8fcda05 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.249920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29893cf-4aa4-4f33-98c8-d8d9e8fcda05 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.250410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29893cf-4aa4-4f33-98c8-d8d9e8fcda05 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.253133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 78/159] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:25.259950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.260605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.261106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.286492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:25.288282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:25.307356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f206c8-1413-4431-afda-c250a4dd958c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:25.347253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:25.347887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.348163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.348430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.359369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:25.364638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:25.365553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:25.366329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 82/160] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:25.372460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:25.395176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 79/161] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:25.475988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:25.480018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7ed94-e22f-4681-9565-c917068d9264 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:25.516155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:25.516773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.517050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.517328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.518003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:25.518302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:25.519819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:25.529877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 83/162] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:25.535641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:25.655378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:25.665294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c047f0-1589-4802-ac17-82c0f67cc6f7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:25.704897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:25.705545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.705824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.706095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] PATH_INFO: `/v3/users/d0a0e6dd3f1445518b190cea2b73f55b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.707260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a0e6dd3f1445518b190cea2b73f55b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:25.708731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:25.710679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 80/163] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:25 2022] GET /identity/v3/users/d0a0e6dd3f1445518b190cea2b73f55b => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:25.716580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:25.848177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6412b-9882-42e7-99d2-17cf93b5bc58 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:25.885496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:25.886108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:25.886439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:25.886809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:25.919899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:25.973906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:25.975491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e7cef-9029-4351-8ac3-bb7defefd3f2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:25.976454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.109588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 84/164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:26.114405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d536-f6ff-4f31-a96b-d583f0ca75d7 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.114830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d536-f6ff-4f31-a96b-d583f0ca75d7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.115701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d536-f6ff-4f31-a96b-d583f0ca75d7 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.116752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 85/165] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:26.119152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4d536-f6ff-4f31-a96b-d583f0ca75d7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.148760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d88d1-7bcc-4685-b980-8bdfaf260698 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.185215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:26.189635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:26.190336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.190680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.190957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.203945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-ee7c92eb-7ae0-4d97-ad95-3bed8390cca3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:26.209188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:26.210049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 81/166] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:26.212337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.214263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.214263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.231956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:26.233170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:26.234432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4d536-f6ff-4f31-a96b-d583f0ca75d7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.248603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:26.271249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:26.272017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.272298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.272741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.273972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:26.275635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:26.277110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 86/167] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:26.277961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.308664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 82/168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:26.310347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.505758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4bbb6-9c5e-4522-b221-8c8afe3cb55e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.514318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567acbfa-f297-4959-9c77-9f2a3ff89796 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.542464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:26.544034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.544514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.545101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.546024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=d0a0e6dd3f1445518b190cea2b73f55b) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:26.546569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:26.548211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:26.554402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 83/169] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/role_assignments?user.id=d0a0e6dd3f1445518b190cea2b73f55b&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:26.581305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:26.593851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:26.594485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.594791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.595068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.595761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:26.596066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:26.597457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:26.617324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 87/170] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:26.622958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.642953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:26.741334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420190-4bce-4b17-9cb0-e192f5ea39ad None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.782768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:26.783748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:26.784242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:26.784734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:26.786187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:26.787909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:26.789763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:26.799678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.852512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:26.960166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:26.983732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee9560-04ce-4a7b-a22b-a0b462a71331 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:26.997103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bcb95-c7af-4f8a-ba51-b0bc6de48e43 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.019972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.021067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.021637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.022106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.035105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:27.041042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:27.041933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:27.044189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:27.046591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 85/172] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:27.051849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:27.057353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.058233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.061166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.061686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.189638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:27.211007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.212829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.273438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b25b2-a1d9-4c4e-b911-bf84eb34af8b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.335921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.336465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.336756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.337011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.337688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:27.337991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.339687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.354625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 86/173] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:27.359375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:27.365009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:27.385536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 88/174] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:27.391261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:27.495315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b0968-fee8-4112-a975-a336dc54c978 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.542931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.544355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.545308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.545308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] PATH_INFO: `/v3/users/27c6e1813c56499ab5b07dc305f8bdc3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.547175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] RBAC: Authorizing `identity:get_user(user_id=27c6e1813c56499ab5b07dc305f8bdc3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.548627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.550201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 87/175] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:27 2022] GET /identity/v3/users/27c6e1813c56499ab5b07dc305f8bdc3 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:27.557574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:27.578861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:27.590755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac1988-c483-45d7-b6b7-e5f1b53b2dd4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.633424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:27.649476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.650120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.650397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.650740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] PATH_INFO: `/v3/projects/5ffc1609ca45470cad1de533f23407af` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.651876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5ffc1609ca45470cad1de533f23407af)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.653938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.655908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 89/176] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:27 2022] GET /identity/v3/projects/5ffc1609ca45470cad1de533f23407af => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:27.661578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:27.687753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86b9cb-304c-42c5-9429-21a526d29f49 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.759940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.760571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.760854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.761134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.770637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797227-8fca-4a8c-b4e0-552dd291c7c0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:27.832759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:27.833377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.833654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.833922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.834682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=5ffc1609ca45470cad1de533f23407af, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:27.834987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.837928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6408-4112-4574-9bfc-0f89dd22e973 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.854267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 90/177] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:27 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=5ffc1609ca45470cad1de533f23407af&role.id=7c78b3797aea4e3280d32765c45d23df => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:27.855326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb128a-4e4e-4166-a843-ff7cd703bf25 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.855641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb128a-4e4e-4166-a843-ff7cd703bf25 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.855915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb128a-4e4e-4166-a843-ff7cd703bf25 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.858894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 91/178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:27.863584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487554-2f53-4f0a-bbfd-77fa1a9f7938 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.863863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487554-2f53-4f0a-bbfd-77fa1a9f7938 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.864134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487554-2f53-4f0a-bbfd-77fa1a9f7938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.899942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1e487554-2f53-4f0a-bbfd-77fa1a9f7938 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:27.901147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1e487554-2f53-4f0a-bbfd-77fa1a9f7938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:27.913648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:27.915261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:27.932677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:27.990339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 92/179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:27.991824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19447b-cb87-4778-aa2d-b2dc5e68c197 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.992111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19447b-cb87-4778-aa2d-b2dc5e68c197 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.992383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19447b-cb87-4778-aa2d-b2dc5e68c197 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:27.993711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 93/180] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:27.999164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:27.999445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:27.999752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.019934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:28.021219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:28.048524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:28.097317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 94/181] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:28.133544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 88/182] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:28.139619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.262983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:28.290609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6d240-f99a-431a-a994-ce11edce6315 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:28.350044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241f760-b853-4385-95aa-64087aad472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:28.350721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241f760-b853-4385-95aa-64087aad472d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.351003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241f760-b853-4385-95aa-64087aad472d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.351276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241f760-b853-4385-95aa-64087aad472d None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.355207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241f760-b853-4385-95aa-64087aad472d None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:28.356742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241f760-b853-4385-95aa-64087aad472d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:28.358801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 95/183] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:28.364515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.365448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9a49-72b0-485c-9010-b69ca5e74e3f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.365680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9a49-72b0-485c-9010-b69ca5e74e3f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.365959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9a49-72b0-485c-9010-b69ca5e74e3f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.367306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 96/184] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:28.374185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a976bf-9954-4a58-a471-9c1345f39fa0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.374469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a976bf-9954-4a58-a471-9c1345f39fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.374790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a976bf-9954-4a58-a471-9c1345f39fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.394975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-64a976bf-9954-4a58-a471-9c1345f39fa0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:28.396294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-64a976bf-9954-4a58-a471-9c1345f39fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:28.474386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 97/185] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:28.479780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915203af-fea7-416a-bf63-bfad291a8e99 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.480075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915203af-fea7-416a-bf63-bfad291a8e99 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.480357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915203af-fea7-416a-bf63-bfad291a8e99 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.482407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 98/186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:28.483216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915203af-fea7-416a-bf63-bfad291a8e99 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.501826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4e311-009f-40fd-9101-ada3f9bc843a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:28.544793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:28.546491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.546959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.547667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.548556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=27c6e1813c56499ab5b07dc305f8bdc3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:28.549160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:28.551401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc29c6-e730-4e09-897b-7adabc826655 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:28.562557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 89/187] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/role_assignments?user.id=27c6e1813c56499ab5b07dc305f8bdc3&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:28.562557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.562557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.562557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.581557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:28.582854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:28.583801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:28.608921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915203af-fea7-416a-bf63-bfad291a8e99 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:28.615151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:28.652494 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:28.653442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.653919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.654360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.663809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 90/188] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:28.664704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.668707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:28.674009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:28.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:28.675755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:28.678062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 99/189] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:28.688258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.818480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13965ff9-27cd-470e-b7ed-c85b9f32fc68 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:28.843939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6d8f7-5eba-4bf2-a582-70e500a3394a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:28.862397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:28.863083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.863360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.863644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.880579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:28.885457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:28.886343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:28.887298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 91/190] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:28.888059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.892193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:28.892651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:28.892932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:28.893203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:28.893882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:28.895080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:28.895686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:28.905035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 100/191] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:28.905783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:28.909434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:28.923056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.000495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b808d-2596-41d4-9168-cfad6abb1dbd None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.009547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3e5e8-a060-4054-9f23-b176c70a22bc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.034973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.035604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.035884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.036157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.037567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.039106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.040666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 92/192] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.042713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.043349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.043642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.043814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.044095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.044815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:29.045341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.046742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.059994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 101/193] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.059994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.121591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.143204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.152632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35755a-70eb-4475-98ea-bea96131bc89 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.160860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e43d05-4e78-4185-b2c9-60e18016816d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.185937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.186595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.186872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.187148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.196080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.196702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.196977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.197244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.200833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.202396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.203933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 93/194] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.206001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.207836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:29.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:29.214351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:29.215481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 102/195] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:29.216348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.308188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8c44-1f69-4596-8428-c41c9ff58197 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.312904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8462d-4687-4c01-a881-31e9ee367fc0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.335740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.343104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.343709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.343990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.344271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.344924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.346373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.347032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.347661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.347940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.348216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] PATH_INFO: `/v3/users/b4316462724642ef93229e65d3c3c5ce` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.349351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] RBAC: Authorizing `identity:get_user(user_id=b4316462724642ef93229e65d3c3c5ce)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.350806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.352577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 103/196] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/users/b4316462724642ef93229e65d3c3c5ce => generated 289 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.353122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.369038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 94/197] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:29 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:29.369925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.380093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.484356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c56713-48ea-49b0-9eea-e17a0e0d4084 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.491538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d0893-6dd9-4c32-8fcf-4b4cec07e40d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.524508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.525262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.525440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.525677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.530696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.531326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.531609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.531885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.532578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:29.532911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.534365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.544692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 95/198] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.546091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.596760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.607549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.609428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.628818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.671679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7116aa-b531-4967-847a-dad832eb987e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.715117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.716014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.716288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.716556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.754728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 104/199] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:29.755353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.805681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.807207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.878549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.897446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d063c-e325-4f76-9b12-1f85f34838e4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:29.903444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:29.917687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 96/200] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:29.923194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:29.942056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:29.942611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:29.942912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:29.943194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:29.944588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:29.946027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:29.947804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 105/201] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:29.948399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.052362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3daba3a-20db-4eb2-8a5e-ccda6678077d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.059572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91185-9ccf-4068-b4e3-1f2f03bd81e9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.100498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.102048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.102551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.103018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.104354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.106392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.107061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.107339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.107597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.107711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.108949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.109844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 97/202] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:30.110824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.116253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.117489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 106/203] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:29 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:30.118129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.154904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:30.170920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:30.233492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72eba-7528-421d-a904-fe34d1c48c55 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.244392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040f6f-d2db-49f0-bf2a-d6b1fb78119f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.269545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.270162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.270439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.270836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.271573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=1881c136aced45f795707f11d6638484) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:30.271879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.273323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.279649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 107/204] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:30 2022] GET /identity/v3/role_assignments?user.id=1881c136aced45f795707f11d6638484&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:30.280528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.290870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.292301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.292797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.294146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.405460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2ce4-fc24-4186-b73f-2f040ebd15f6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.442967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:30.451622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.452695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.455825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.456421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.457697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24, user.id=b4316462724642ef93229e65d3c3c5ce) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:30.458016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.461612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.467902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.469446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.504678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 108/205] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:30 2022] GET /identity/v3/role_assignments?user.id=b4316462724642ef93229e65d3c3c5ce&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:30.510924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:30.584464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 98/206] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:30.590331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.785354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6e44f-3465-4b8a-8b7c-180515389fcf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:30.841374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:30.842015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:30.842287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:30.842584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:30.843753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:30.845196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f6979-6f12-4ab7-8beb-46ba1c14ac94 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:30.847154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 109/207] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:30 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:30.855650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:30.965481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87c65-e3b1-40d4-872d-fef829945b94 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:31.007858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:31.008486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:31.008768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:31.009039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:31.009785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=1881c136aced45f795707f11d6638484) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:31.010082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:31.011598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee7f3-18f1-4777-90d9-22ad53c299a7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:31.017515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 99/208] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:30 2022] GET /identity/v3/role_assignments?user.id=1881c136aced45f795707f11d6638484&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:31.018015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:31.884731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a71fb9-c117-4f8c-a717-6c2aedcaf003 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:31.885734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a71fb9-c117-4f8c-a717-6c2aedcaf003 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:31.886238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a71fb9-c117-4f8c-a717-6c2aedcaf003 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:31.888324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 110/209] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:31.893790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:31.894205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:31.894390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:31.913791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:31.915301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:31.923862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22de1-a25f-4993-9c3d-c32f81ba2f40 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:31.924168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22de1-a25f-4993-9c3d-c32f81ba2f40 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:31.924425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22de1-a25f-4993-9c3d-c32f81ba2f40 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:31.925821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 111/210] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:31.931954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:31.932253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:31.932520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:31.952112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:31.953323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:31.972275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:31.993154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 100/211] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:32.029316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 112/212] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:32.284632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.300053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.395430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a69ae-607b-4143-aa34-bf0bab454c73 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:32.416723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf02d0-7fc0-493d-9f56-92f5bf4a0fae None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:32.425635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:32.432342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:32.432981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.433255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.434164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.435395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:32.436949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:32.438643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 101/213] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:32.447807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.457779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:32.458498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.458888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.459240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.471492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:32.476821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:32.477890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:32.478900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:32.485387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 113/214] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:32.486505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.504128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:32.591211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc64db-b01c-45e2-9684-d015297eced7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:32.629878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8d72-3fb0-4045-a79b-fe3bc7eaf5b6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:32.632156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:32.632766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.633038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.633305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.634002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:32.635478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf671db-37ff-4ed6-87ba-bfcb4ae07f57 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:32.661286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 102/215] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:32 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:32.669090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:32.669780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.670065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.670338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.671059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:32.671347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8c6e-0239-42aa-a7e9-768e94babddd None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.671442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:32.671638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8c6e-0239-42aa-a7e9-768e94babddd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.671916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8c6e-0239-42aa-a7e9-768e94babddd None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.672834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:32.674706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 103/216] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:32.679413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.679991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.680481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.685269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 114/217] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:32.691147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.701894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:32.703476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:32.773914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:32.787639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:32.808260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 104/218] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:32.821018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5432e-4e9c-432a-bd45-879c57431bfc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:32.863021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:32.863595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:32.863879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:32.864161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] PATH_INFO: `/v3/users/891cad8dc6044c3583ced1e52761c690` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:32.879086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] RBAC: Authorizing `identity:get_user(user_id=891cad8dc6044c3583ced1e52761c690)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:32.880550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:32.882230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 115/219] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:32 2022] GET /identity/v3/users/891cad8dc6044c3583ced1e52761c690 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:32.888066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:32.999924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7ae76-9ea1-4a6a-8f48-123a11777ec0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.039335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.039959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.040243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.040517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.089642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.107225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:33.134390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:33.136353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1af8a-c2a2-407b-a1e3-ed2ccd33344e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:33.233345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b5e36-8615-4b2c-bcac-26e916d846a7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.274128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 105/220] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:33.280376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ba267-2793-48f3-8e4c-7687461e0043 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.280376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ba267-2793-48f3-8e4c-7687461e0043 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.281597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ba267-2793-48f3-8e4c-7687461e0043 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.283255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 106/221] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:33.284182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ba267-2793-48f3-8e4c-7687461e0043 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.284621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.285481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.285949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.286408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.290022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:33.303914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3c436618-6ab2-464c-8be8-6869ce654844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:33.310710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:33.311591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:33.312410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:33.314915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 116/222] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:33.320101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.320101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.320101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.345705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:33.347817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:33.390469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:33.415668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ba267-2793-48f3-8e4c-7687461e0043 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.443070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 117/223] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:33.443937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.461580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.462639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.463114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.463884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.465484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:33.467298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:33.469584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 107/224] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:33.475816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.628744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d69e63-a517-4052-82ea-04762aaa93b9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.654200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5e725-e5b8-487b-b921-f79987ac5499 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.695457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:33.697604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.698224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.698494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.698780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.699811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=891cad8dc6044c3583ced1e52761c690) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:33.700119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:33.701685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:33.715638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.716255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.716532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.716847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.717260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 108/225] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/role_assignments?user.id=891cad8dc6044c3583ced1e52761c690&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=7c78b3797aea4e3280d32765c45d23df => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:33.717593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:33.717896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:33.721085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:33.725189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.739689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 118/226] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:33.740601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.762894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:33.928864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d0f9-c1ed-4292-8f0c-e430c08c5238 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.934812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077507ca-24f0-4ba5-bc59-2d50357d877f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:33.973054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.973913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.974350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.974786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.975948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:33.976944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:33.977139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:33.977374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:33.978954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:33.980983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:33.983266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 119/227] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:33.988643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:33.993377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:34.008145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:34.009499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:34.010300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 109/228] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:34.017863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:34.069475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:34.150242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f83d1e-5a85-4c19-b549-a8be31e7c0d7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:34.164826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:34.206225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33c432-d213-4555-9dc3-a0dbe6c3dd4b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:34.217653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:34.218300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:34.218599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:34.218871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:34.220003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:34.221455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:34.224039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 120/229] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:33 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:34.229476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:34.278367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:34.279008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:34.279282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:34.279554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:34.280224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:34.280523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:34.281913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:34.303162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 110/230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:34.307108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:34.400339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92123ac-e214-4b61-b746-e70827099896 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:34.431094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:34.443157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:34.443826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:34.444481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:34.444768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/users/1881c136aced45f795707f11d6638484/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:34.446863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, user_id=1881c136aced45f795707f11d6638484, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:34.448292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:34.473549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 121/231] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:34 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/users/1881c136aced45f795707f11d6638484/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:39:34.486607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdd0fa-4547-4faa-ab52-8f73c713d0c3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:34.523988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:34.524714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:34.525009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:34.525283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:34.526625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:34.528773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:34.530775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 111/232] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:34 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:34.536151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:34.700619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ae73c-a1b1-4200-858f-1653dd6ede7d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:34.746458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:34.747288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:34.747771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:34.748211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:34.749658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:34.751133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97554739-bb75-420c-a95c-ee8cea98b1d4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:34.753133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 122/233] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:34 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:34.765046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:34.782766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:34.940477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98b978-b98d-4512-891b-56a93c827feb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:35.004602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:35.005207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:35.005491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:35.006192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/users/1881c136aced45f795707f11d6638484/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:35.008394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, user_id=1881c136aced45f795707f11d6638484, role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:35.009868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81848369-9d2b-4799-b325-13e16ac44869 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:35.026536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 112/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:34 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/users/1881c136aced45f795707f11d6638484/roles/b42a9d0451fc419ab1547c88f1248317 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:35.349293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fdb3-a9d8-4bbc-8b80-fc3954cdf126 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:35.349293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fdb3-a9d8-4bbc-8b80-fc3954cdf126 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:35.349293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fdb3-a9d8-4bbc-8b80-fc3954cdf126 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:35.350274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 123/235] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:35.356112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14bc41-86c5-4e06-8372-8afd3f48d8d4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:35.356702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14bc41-86c5-4e06-8372-8afd3f48d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:35.357190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14bc41-86c5-4e06-8372-8afd3f48d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:35.377799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2f14bc41-86c5-4e06-8372-8afd3f48d8d4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:35.379346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2f14bc41-86c5-4e06-8372-8afd3f48d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:35.449664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:35.465407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 113/236] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:35.952460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fdb3-a9d8-4bbc-8b80-fc3954cdf126 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:35.954813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37006f-45e6-4d0e-93ae-f95b4874fb22 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:35.955284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37006f-45e6-4d0e-93ae-f95b4874fb22 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:35.955723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37006f-45e6-4d0e-93ae-f95b4874fb22 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:35.957233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 114/237] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:35.971732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:35.972005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:35.972271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.006545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:36.007796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:36.075085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:36.105840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 115/238] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:36.158696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fdb3-a9d8-4bbc-8b80-fc3954cdf126 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:36.219962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:36.220512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.220791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.221063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.234190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:36.239488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:36.240521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:36.241431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:36.243694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 124/239] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:36.250105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:36.361613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514efeb-e0c1-4a81-b895-9673c6d96742 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:36.390987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:36.409558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:36.410457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.411011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.411429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.412284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:36.412739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:36.416432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c13fa1-e617-4bbb-9386-02ec564a0284 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:36.427808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 116/240] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:36.436655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:36.567241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcd8f-20d4-4c81-82ed-2fb9808e6099 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.567731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcd8f-20d4-4c81-82ed-2fb9808e6099 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.568147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcd8f-20d4-4c81-82ed-2fb9808e6099 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.569765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 117/241] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:36.574980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.575584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.576003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.597805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:36.599266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:36.608662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dde7a-b06f-48c8-836f-d3de090c731b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:36.620407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:36.646960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:36.647901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.648383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.648885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] PATH_INFO: `/v3/users/57c576e1330c45178ff2aa0107e83cd7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.663398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] RBAC: Authorizing `identity:get_user(user_id=57c576e1330c45178ff2aa0107e83cd7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:36.665096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:36.667290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 125/242] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:36 2022] GET /identity/v3/users/57c576e1330c45178ff2aa0107e83cd7 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:36.673558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:36.682143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 118/243] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:36.687130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:36.812584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb591455-8b09-4b39-b0a7-3554e09081c6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:36.835047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:36.850806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:36.851802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.852283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.852756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.901376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cb53c-46f2-4e0c-8e15-dff950a5e7a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:36.928455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:36.930379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:36.937420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:36.938060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:36.938375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:36.938824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:36.940094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:36.941547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:36.943628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 126/244] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:36.951423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.002822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.071025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 119/245] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:37.076552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8021d3-205d-40b8-9d7d-96fdbdd9a0a6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.077241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.097337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.125423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.126406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.126836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.127247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.128036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:37.129714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:37.154662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 127/246] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:36 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:37.155867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.232864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db62910-e6b2-4b27-bb0d-0fa25c3bea4f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.269245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.282226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864a787-75da-491f-a8ce-a270ab8de157 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.321838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.322973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.323462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.324512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.326829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:37.328611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:37.330637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 128/247] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:37.337256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.343432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.345553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.347929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.348476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.375899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:37.380827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:37.381726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:37.382565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:37.386801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 120/248] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:37.398279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.491214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d46d6-7d7f-4282-9263-9bdaa5a50bfa None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.512462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.538213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.538843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.539128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.539407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.540248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=57c576e1330c45178ff2aa0107e83cd7) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:37.540563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:37.542067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465d537-90a3-421d-85d6-5de10b9b609d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:37.544144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec21882-96ce-4d8e-aab9-91070a1bbd38 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.550425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 129/249] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/role_assignments?user.id=57c576e1330c45178ff2aa0107e83cd7&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:37.585222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.586108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.586696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.587108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.587934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:37.588631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:37.590637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:37.596058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd26c78-93e5-4ed6-bf4e-324b7e1ecbe3 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.596486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd26c78-93e5-4ed6-bf4e-324b7e1ecbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.596868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd26c78-93e5-4ed6-bf4e-324b7e1ecbe3 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.599597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 130/250] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:37.601209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 121/251] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:37.604302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.604712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.605252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.607422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.638500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:37.639669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:37.668124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.748267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 131/252] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:37.760465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776386e3-e451-4670-a9a0-da44eee54add None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.804267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.804914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.805259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.805536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] PATH_INFO: `/v3/users/891cad8dc6044c3583ced1e52761c690` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.806789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] RBAC: Authorizing `identity:get_user(user_id=891cad8dc6044c3583ced1e52761c690)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:37.808721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e0371-c7fc-4c81-b8d6-9c1a64702a82 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:37.810422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 122/253] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/users/891cad8dc6044c3583ced1e52761c690 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:37.815991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:37.841952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e61-7bd1-4109-b248-5c2dbb342364 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.842248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e61-7bd1-4109-b248-5c2dbb342364 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.842587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e61-7bd1-4109-b248-5c2dbb342364 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.844124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 123/254] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:37.849450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.849782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.850051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.870203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:37.872019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:37.874319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:37.955094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871e9a-5f85-4feb-83ed-2ab3fdf78180 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:37.959446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 124/255] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:37.995329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:37.996323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:37.996775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:37.997239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:37.998577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:38.000192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:38.002296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 132/256] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:37 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:38.007987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.131876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.153284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9ef1d-ee1d-432f-9ea2-b667976e30df None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.176916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:38.225129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.225800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.226082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.226351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/users/891cad8dc6044c3583ced1e52761c690/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.235298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, user_id=891cad8dc6044c3583ced1e52761c690, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:38.236770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:38.252288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:38.259302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 125/257] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:38 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/users/891cad8dc6044c3583ced1e52761c690/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:38.348406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f5845-af46-4bd4-9792-a2049f68b894 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.377296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.409782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.410728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.411015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.411291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.443606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:38.449178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:38.450116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:38.450942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 133/258] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:38.457717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.526245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62092d0-2f94-48b2-8fea-24aa87b53906 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.566152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.566990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.567280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.567602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.580203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:38.585343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:38.586218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:38.587102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 126/259] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:38.594626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.661619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:38.679807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:38.715166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e75fb0-6f67-47f8-9b2f-e6084e8b28e6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.732930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c583b85-601e-4f0f-9ed1-c542a92a734f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.754263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.754972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.755254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.755530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.756292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:38.756598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:38.758100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:38.768079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 127/260] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:38.773700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.819395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.820039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.820320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.820597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.821290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:38.821596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:38.823020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:38.841751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 134/261] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:38.847501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:38.890615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222509-e016-4c1e-b34f-a22173282cd1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:38.913285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:38.932391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:38.933609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:38.934066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:38.934439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:38.935841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:38.937381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:38.939327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:38.945135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:39.057062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a47f4-a283-4200-9334-c44f554789c2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:39.101211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:39.101832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:39.102113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:39.102385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:39.112687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1979-4cbf-4f25-815a-6b82fba68707 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:39.168638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:39.184853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:39.186459 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:39.211233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:39.211887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:39.212174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:39.212446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] PATH_INFO: `/v3/users/1881c136aced45f795707f11d6638484` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:39.213887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] RBAC: Authorizing `identity:get_user(user_id=1881c136aced45f795707f11d6638484)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:39.215472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:39.217236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 135/263] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/users/1881c136aced45f795707f11d6638484 => generated 283 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:39.222750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:39.253337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:39.304813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 129/264] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:39:39.311517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:39.409655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:39.426131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89feb-fdaf-4ae6-aae4-80896a195296 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:39.468355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:39.469011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:39.469344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:39.469624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:39.470668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519d05-e628-4d1d-8949-7721518c4eda None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:39.470801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:39.472362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:39.474373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 130/265] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:39 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:39.480658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:39.552096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:39.553237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:39.553718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:39.554182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:39.592113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db50731-5110-4eb3-971e-d2d0b9155a85 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:39.647789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:39.648465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:39.648688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:39.648963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:39.649708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=1881c136aced45f795707f11d6638484) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:39.650016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:39.651491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:39.663686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 131/266] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:39 2022] GET /identity/v3/role_assignments?user.id=1881c136aced45f795707f11d6638484&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:39.679464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:39.681032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:39.708844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:39.834429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 136/267] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:39:39.843155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:39.880393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:39.984743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e536e36-ba7a-45a3-8948-bd575e94af4e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:40.033538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:40.034176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.034455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.034744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.039199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:40.040664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea0882-ec75-47b3-8003-5ae7042d9dca None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:40.043094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 132/268] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:39 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:40.048858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:40.194304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db647-244a-4730-986b-96548bffb2ef None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:40.211395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:40.248968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:40.249566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.249848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.250125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.250954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=1881c136aced45f795707f11d6638484) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:40.251262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:40.252713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becc15e-5a21-43cf-8a1b-5f8c88ee0b60 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:40.261790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 137/269] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:40 2022] GET /identity/v3/role_assignments?user.id=1881c136aced45f795707f11d6638484&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:40.617643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41c85-0d5c-4c49-9565-756cc702d44d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.620283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41c85-0d5c-4c49-9565-756cc702d44d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.620283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41c85-0d5c-4c49-9565-756cc702d44d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.620283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 133/270] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:40.627004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.627295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.627577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.649204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:40.650406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:40.690540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17438-a4e5-494b-b9bb-34e4a1f11a5a None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.690540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17438-a4e5-494b-b9bb-34e4a1f11a5a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.690540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe17438-a4e5-494b-b9bb-34e4a1f11a5a None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.691849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 134/271] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:40.699141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92945ab-fe0a-4c68-b574-d54a91185490 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:40.699857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92945ab-fe0a-4c68-b574-d54a91185490 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:40.700326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92945ab-fe0a-4c68-b574-d54a91185490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:40.723169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b92945ab-fe0a-4c68-b574-d54a91185490 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:40.725644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b92945ab-fe0a-4c68-b574-d54a91185490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:40.747968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 138/272] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:40.809832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 135/273] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:41.040659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.069533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b176a-2421-4903-af29-d391e017cab3 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.070171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b176a-2421-4903-af29-d391e017cab3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.070703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b176a-2421-4903-af29-d391e017cab3 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.072125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 136/274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:41.077143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.077415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.077689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.102711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:41.103888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:41.117951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.156978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3df3c-fedb-43f7-9e37-29b7c21c8790 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.165437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.173436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 137/275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:41.175672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.196079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.196943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.197407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.197853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.209569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:41.215522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:41.216393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:41.217201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:41.219803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 139/276] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:41.225107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.315022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1653e2-72f9-4be7-b6d7-9e299a1d515b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.351274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261a4a-bbf3-48e7-87d8-489209cf618a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.354234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.355341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.355884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.356731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.369764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:41.376087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:41.377046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:41.377925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:41.380421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 138/277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:41.386939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.392259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.393591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.393591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.393591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.394185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:41.394497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.395905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.402626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.405794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 140/278] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:41.412856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.453861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.507185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebdad8-43e0-4b0c-8c60-736088665817 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.526823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a4f16-8388-4c82-a834-5e69067d9e84 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.539606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.540217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.540496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.540773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.541447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:41.541754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.544059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.552255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 139/279] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:41.553064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.564284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.565015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.565349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.565577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] PATH_INFO: `/v3/users/57c576e1330c45178ff2aa0107e83cd7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.566763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] RBAC: Authorizing `identity:get_user(user_id=57c576e1330c45178ff2aa0107e83cd7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.568203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6633-b2f5-480e-a795-af7920273459 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.569790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 141/280] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/users/57c576e1330c45178ff2aa0107e83cd7 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:41.570938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d00b3-c05f-466b-bc3b-f3cf11fa9785 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.571222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d00b3-c05f-466b-bc3b-f3cf11fa9785 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.571502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d00b3-c05f-466b-bc3b-f3cf11fa9785 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.572993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 142/281] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:41.577497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d00b3-c05f-466b-bc3b-f3cf11fa9785 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.628294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.667827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78ba-2ff6-444d-a61a-d8818119937b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.676181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.683569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d00b3-c05f-466b-bc3b-f3cf11fa9785 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.702503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.703155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.703434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.703704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.704819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.706286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.708067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 140/282] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:41.708623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.721331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.721960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.722242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.722527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] PATH_INFO: `/v3/users/f44b0b038ec842479174dc686a3fde96` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.735980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] RBAC: Authorizing `identity:get_user(user_id=f44b0b038ec842479174dc686a3fde96)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.737414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9c1ac-0280-41ad-a441-ade2c8558915 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.739189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 143/283] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/users/f44b0b038ec842479174dc686a3fde96 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:41.740297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.740584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.740858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.758580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:41.759795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:41.811342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ebf0e-51a3-4d58-b888-a4feb01cbccf None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.825479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 144/284] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:41.829900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.845300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.845933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.846217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.846491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.847637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.849097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:41.850907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 141/285] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:41.855964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:41.868470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.922723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:41.940101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7c9bc-b971-433d-9f05-7870e69d9c50 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.970729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a458-1fd5-45da-9220-46fed323de4d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:41.977342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:41.978353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:41.978742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:41.979021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:41.979708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:41.981832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:42.004312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 145/286] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:39:41 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:42.005103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:42.006774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.007391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.007676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.007952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.009520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, user.id=f44b0b038ec842479174dc686a3fde96, scope.system=all) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:42.009520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:42.010642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:42.024652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 142/287] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 12 11:39:41 2022] GET /identity/v3/role_assignments?user.id=f44b0b038ec842479174dc686a3fde96&scope.system=all&role.id=7c78b3797aea4e3280d32765c45d23df => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:42.127436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:42.160182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686e68-73c6-486f-95fe-f6214c982ffe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:42.216378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.217069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.217356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.217647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/users/57c576e1330c45178ff2aa0107e83cd7/roles/0b8995b146864249aeeb7d42c6f75d7e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.219691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, user_id=57c576e1330c45178ff2aa0107e83cd7, role_id=0b8995b146864249aeeb7d42c6f75d7e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:42.221164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:42.226360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:42.236029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:42.241413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 146/288] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:39:42 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/users/57c576e1330c45178ff2aa0107e83cd7/roles/0b8995b146864249aeeb7d42c6f75d7e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:42.272065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90888d6c-04fb-4a72-b0cd-7797d19a973d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:42.314647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.315306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.315586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.315861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.328719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:42.334307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:42.335344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:42.336141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 143/289] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:42.342500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:42.473110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec1570-e64b-4b60-9fc6-7a50f309732b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:42.511975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.512983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.513496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.513954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.514874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:42.515369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:42.516933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:42.527542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 147/290] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:42.532769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:42.647324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e934-303b-423c-a869-0621e6654475 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:42.677334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:42.687963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.688614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.688898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.689174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] PATH_INFO: `/v3/users/891cad8dc6044c3583ced1e52761c690` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.690336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] RBAC: Authorizing `identity:get_user(user_id=891cad8dc6044c3583ced1e52761c690)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:42.691833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:42.693585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 144/291] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:42 2022] GET /identity/v3/users/891cad8dc6044c3583ced1e52761c690 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:42.699012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:42.802354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:42.858144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb9a0-ebfb-48b6-83dd-7ce352106b4f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:42.903791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:42.904408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:42.904690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:42.904962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:42.998705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:43.000192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:43.084795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:43.149242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 148/292] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:43.154732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:43.308918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f3e24-4f37-48be-b1ad-ec8ac9985062 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:43.344190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:43.344860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:43.345141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:43.345415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:43.346682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:43.348276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b64d83-76c9-4a66-b65b-2477579536dd None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:43.350409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 145/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:43 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:43.356591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:43.457963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbe821-da71-4a39-b721-76f1bcabc42f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:43.496091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:43.496773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:43.497148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:43.497418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:43.498337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=891cad8dc6044c3583ced1e52761c690) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:43.498848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:43.501086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ac667-002f-4681-929d-a728360a5bdc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:43.513419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 149/294] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:43 2022] GET /identity/v3/role_assignments?user.id=891cad8dc6044c3583ced1e52761c690&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=7c78b3797aea4e3280d32765c45d23df => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:44.541543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5377d-7cfb-4ef1-8ae4-456d855b94e1 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.542573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5377d-7cfb-4ef1-8ae4-456d855b94e1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.542851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5377d-7cfb-4ef1-8ae4-456d855b94e1 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.545084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 146/295] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:44.551055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f73646-4f5d-4cb5-b7a5-26f3ad503b64 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.551055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f73646-4f5d-4cb5-b7a5-26f3ad503b64 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.551055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f73646-4f5d-4cb5-b7a5-26f3ad503b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.570946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-34f73646-4f5d-4cb5-b7a5-26f3ad503b64 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:44.572461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34f73646-4f5d-4cb5-b7a5-26f3ad503b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:44.594253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:44.644670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 150/296] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:44.763217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d451d-9227-4269-afa9-ebbd0df214fe None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.763538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d451d-9227-4269-afa9-ebbd0df214fe None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.763772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d451d-9227-4269-afa9-ebbd0df214fe None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.765226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 147/297] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:44.771309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4380cfc-afdd-4706-a1dc-2fd0ee0211d6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.771607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4380cfc-afdd-4706-a1dc-2fd0ee0211d6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.771873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4380cfc-afdd-4706-a1dc-2fd0ee0211d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.792208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e4380cfc-afdd-4706-a1dc-2fd0ee0211d6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:44.793345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e4380cfc-afdd-4706-a1dc-2fd0ee0211d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:44.797440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043be74-a71c-4470-b5fb-f6c88e118f2e None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.797721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043be74-a71c-4470-b5fb-f6c88e118f2e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.798004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043be74-a71c-4470-b5fb-f6c88e118f2e None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.799693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 148/298] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:44.805255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab34089-2bc8-4124-bab2-ab77d6a1f7bb None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.805541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab34089-2bc8-4124-bab2-ab77d6a1f7bb None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.805817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab34089-2bc8-4124-bab2-ab77d6a1f7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.825547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3ab34089-2bc8-4124-bab2-ab77d6a1f7bb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:44.827826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3ab34089-2bc8-4124-bab2-ab77d6a1f7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:44.883241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 151/299] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:44.898666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:44.905195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cf370-94f7-4bb8-98a1-e9888859ab10 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.906116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cf370-94f7-4bb8-98a1-e9888859ab10 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.906788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cf370-94f7-4bb8-98a1-e9888859ab10 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.907582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 149/300] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:44.908992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 152/301] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:44.917717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:44.918005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:44.918286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:44.929763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cf370-94f7-4bb8-98a1-e9888859ab10 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:44.936945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:44.938140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:45.010855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 150/302] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:45.039273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.051181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cf370-94f7-4bb8-98a1-e9888859ab10 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.085839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.086479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.086798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.087076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.098785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:45.103698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:45.105153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:45.105967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 153/303] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:45.113070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.191765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.221373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a095-293b-4262-9938-5ebcf34c876a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.249111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.257751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.258352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.258658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.258935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.259602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:45.259911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.261330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.274197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 151/304] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:45.275027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.301871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3014d-e9dd-417d-bb81-cfb09518e261 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.336570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.337202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.337479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.337753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.339299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.340758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.342335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 154/305] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:45.342335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.358476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.381490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c868-20fd-4fb0-aa87-18067e7202f9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.414021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.414673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.414951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.415220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.425440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:45.430656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:45.431725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:45.433219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 152/306] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:45.435056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.443839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820425c-a68a-46c1-bf80-92143a91e1e8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.478095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.478863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.479147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.479933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] PATH_INFO: `/v3/users/f44b0b038ec842479174dc686a3fde96` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.481081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] RBAC: Authorizing `identity:get_user(user_id=f44b0b038ec842479174dc686a3fde96)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.482568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.483551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.484850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 155/307] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/users/f44b0b038ec842479174dc686a3fde96 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:45.485366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.536261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dd79e-9d7c-419c-8f51-5f2466949884 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.566999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.571319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.571975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.572253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.572522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.584148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:45.586629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1cb0e-ce82-4ba0-9100-0eff29c85363 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:45.589789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:45.590836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 153/308] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:45.591688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.627192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.627804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.628087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.628364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] PATH_INFO: `/v3/groups` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.629000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.630441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.646215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 156/309] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Sep 12 11:39:45 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:39:45.647029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.702309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d761d11-231f-42bd-9a94-80c885f973ac None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.712411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.739964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.740691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.740982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.741254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.741978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:45.742294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.743731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.754211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 154/310] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:45.755006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.772009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ea07-f2bb-4bd3-a60c-683a64107373 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.811576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.812656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.813082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.813503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] PATH_INFO: `/v3/system/users/f44b0b038ec842479174dc686a3fde96/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.815146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f44b0b038ec842479174dc686a3fde96, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.816933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.826089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 157/311] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Sep 12 11:39:45 2022] PUT /identity/v3/system/users/f44b0b038ec842479174dc686a3fde96/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:45.826908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:45.837524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:45.890379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad715e11-2f39-42e7-87ed-c887b559abed None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.960733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:45.961656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:45.962082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:45.962697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:45.965239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:45.965720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:45.968910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:45.970140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc76033-56af-447d-9e49-9c76077a582a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:45.982643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 155/312] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:45.987839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:46.012822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.014383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.014721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.015067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] PATH_INFO: `/v3/users/57c576e1330c45178ff2aa0107e83cd7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.016295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] RBAC: Authorizing `identity:get_user(user_id=57c576e1330c45178ff2aa0107e83cd7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.017781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.019595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 158/313] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/users/57c576e1330c45178ff2aa0107e83cd7 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:46.025356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:46.029996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:46.126275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e129c9-b25d-4442-92c2-6ab5df207830 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:46.138573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:46.164241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.164976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.165261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.165542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] PATH_INFO: `/v3/users/ee4f8e5b76924164a6224bf55b92b41d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.173418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ceceb-b9fd-416a-bf00-a9484c1951fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:46.180767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] RBAC: Authorizing `identity:get_user(user_id=ee4f8e5b76924164a6224bf55b92b41d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.182310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.184695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 156/314] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:45 2022] GET /identity/v3/users/ee4f8e5b76924164a6224bf55b92b41d => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:46.190020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:46.221810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.223077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.224009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.224909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.297331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:46.299037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.300265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.308926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4ce3c-cd6f-43eb-af98-5c8a3bc3e081 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:46.345662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.346633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.347107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.347602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.348598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, user.id=ee4f8e5b76924164a6224bf55b92b41d, scope.system=all) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:46.349180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.350741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.363045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 157/315] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 12 11:39:46 2022] GET /identity/v3/role_assignments?user.id=ee4f8e5b76924164a6224bf55b92b41d&scope.system=all&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:46.387310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:46.419709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 159/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:46.425824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:46.542128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bf58-5b6e-4000-ab58-5406456f7b23 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:46.579333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.580791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.581269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.581729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.583093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.584747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767088d1-86b3-439a-ac06-10ca03b5868c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.586675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 158/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:46 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:46.593435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:46.705569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef53b2-32f6-4fbe-a93f-ffe8871d384d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:46.741079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:46.741693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:46.741972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:46.742245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:46.743001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, scope.project.id=17e6c4470b9442d68be311b25262e42d, user.id=57c576e1330c45178ff2aa0107e83cd7) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:46.743307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:46.744746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d1b4-0971-45e5-819c-48948c592a7f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:46.751482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 160/318] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:39:46 2022] GET /identity/v3/role_assignments?user.id=57c576e1330c45178ff2aa0107e83cd7&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:47.955430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c087-8722-426a-ad26-291f78e7c44c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:47.955990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c087-8722-426a-ad26-291f78e7c44c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:47.955990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c087-8722-426a-ad26-291f78e7c44c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:47.957316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 159/319] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:47.962586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:47.963229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:47.963388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:47.981790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:47.983007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:48.013468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:48.038358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73f61-3144-4e30-8c38-029f074cb4cd None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.038884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73f61-3144-4e30-8c38-029f074cb4cd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.039148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb73f61-3144-4e30-8c38-029f074cb4cd None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.040620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 160/320] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:48.045858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.046138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.046408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.058441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 161/321] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:48.066019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:48.067233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:48.088485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:48.141182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 161/322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:48.320755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.408953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.422744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99904e2-9e38-4bcf-946d-f5e70ef2a79b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:48.460870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:48.461536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.461819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.462100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.463641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:48.465373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:48.467126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 162/323] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:48.472884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.509816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:48.523078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99840b19-46d0-4681-a9cb-86d8fccca44b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:48.560224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:48.560852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.561129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.561400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.571955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-23bcc030-a03d-49b0-82a9-497c09531a00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:48.577045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:48.577953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 162/324] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:48.583822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493091c8-49ae-4a2c-8db0-f23128fc43c6 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.585063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493091c8-49ae-4a2c-8db0-f23128fc43c6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.585063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493091c8-49ae-4a2c-8db0-f23128fc43c6 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.585063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 163/325] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:48.585245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bbac-3b27-4f9c-8537-7e330f57acc6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:48.585965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.586247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.586534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.606147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:48.607291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:48.614320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:48.617341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:48.617957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.618238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.618531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] PATH_INFO: `/v3/groups` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.619193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:48.620646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:48.634529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 163/326] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Sep 12 11:39:48 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:39:48.635347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.691473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 164/327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:48.742916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:48.751193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c820d-9002-47fc-a2a4-94b69e75aafe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:48.788195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:48.789213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.789693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.790163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.791110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:48.791631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:48.793944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:48.803328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 164/328] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:48.809629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.918678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc6238-199c-4cb3-82ed-ad1fda165284 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:48.954368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837b915-f346-4834-98f3-169103522871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:48.955122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837b915-f346-4834-98f3-169103522871 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:48.955402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837b915-f346-4834-98f3-169103522871 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:48.955680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837b915-f346-4834-98f3-169103522871 None admin] PATH_INFO: `/v3/users/f44b0b038ec842479174dc686a3fde96` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:48.956903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837b915-f346-4834-98f3-169103522871 None admin] RBAC: Authorizing `identity:get_user(user_id=f44b0b038ec842479174dc686a3fde96)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:48.958375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837b915-f346-4834-98f3-169103522871 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:48.960170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 165/329] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/users/f44b0b038ec842479174dc686a3fde96 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:48.962868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.965987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837b915-f346-4834-98f3-169103522871 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:48.979885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:49.095962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837b915-f346-4834-98f3-169103522871 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:49.101746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f853f7-005c-4d02-9cc8-cb7c2412ed71 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:49.148355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:49.148968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:49.149240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:49.149505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:49.150240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] RBAC: Adding query filter params (role.id=7c78b3797aea4e3280d32765c45d23df, user.id=f44b0b038ec842479174dc686a3fde96, scope.system=all) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:49.150567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:49.152142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:49.157037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:49.157945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:49.158369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:49.158809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:49.166570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 166/330] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/role_assignments?user.id=f44b0b038ec842479174dc686a3fde96&scope.system=all&role.id=7c78b3797aea4e3280d32765c45d23df => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:49.171366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:49.176417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:49.177355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:49.178153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 165/331] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:49.184293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:49.263525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:49.303828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0710c-e66e-46fa-b47d-a750d10b1ab5 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:49.345372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:49.345994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:49.346268 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:49.346557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:49.347244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:49.347549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:49.349080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:49.362383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 167/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:49.368104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:49.391657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:49.479918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208b6d-1df9-42d2-b17c-86c67d66f2ed None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:49.517091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:49.517736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:49.518027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:49.518292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] PATH_INFO: `/v3/users/ee4f8e5b76924164a6224bf55b92b41d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:49.519478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] RBAC: Authorizing `identity:get_user(user_id=ee4f8e5b76924164a6224bf55b92b41d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:49.521006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e67511-2bf0-45ee-9c27-b5c8589aa60f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:49.522626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 166/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:49 2022] GET /identity/v3/users/ee4f8e5b76924164a6224bf55b92b41d => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:49.528924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:49.644237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84999ce-3b63-45c7-a9a2-00c65178c25a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:49.676217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:49.683762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:49.684430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:49.684779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:49.685059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] PATH_INFO: `/v3/system/users/ee4f8e5b76924164a6224bf55b92b41d/roles/0b8995b146864249aeeb7d42c6f75d7e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:49.686623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee4f8e5b76924164a6224bf55b92b41d, role_id=0b8995b146864249aeeb7d42c6f75d7e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:49.688029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512728b-61be-4ec4-ac07-ca82d303c7df None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:49.701238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 168/334] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Sep 12 11:39:49 2022] PUT /identity/v3/system/users/ee4f8e5b76924164a6224bf55b92b41d/roles/0b8995b146864249aeeb7d42c6f75d7e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:51.130870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ff89-4ea9-49f6-a6d0-4884972ac240 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:51.131353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ff89-4ea9-49f6-a6d0-4884972ac240 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:51.131426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ff89-4ea9-49f6-a6d0-4884972ac240 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:51.132807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 167/335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:51.138118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d93ba-6a3a-41a3-af85-d01625f9207e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:51.138949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d93ba-6a3a-41a3-af85-d01625f9207e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:51.139394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d93ba-6a3a-41a3-af85-d01625f9207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:51.161935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-572d93ba-6a3a-41a3-af85-d01625f9207e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:51.163489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-572d93ba-6a3a-41a3-af85-d01625f9207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:51.252126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 169/336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:51.394307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd07b09-519b-4759-8f2a-62531032bd1e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:51.394865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd07b09-519b-4759-8f2a-62531032bd1e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:51.395334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd07b09-519b-4759-8f2a-62531032bd1e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:51.397142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 168/337] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:51.402868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:51.403379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:51.403947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:51.430791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:51.432389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:51.520296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 170/338] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:51.746237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd07b09-519b-4759-8f2a-62531032bd1e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:51.862873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:51.873516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd07b09-519b-4759-8f2a-62531032bd1e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:51.919435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:51.920103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:51.920377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:51.920643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:51.937747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:51.942810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:51.943731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:51.944512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:51.946851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 169/339] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:51.951434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.021268 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:52.055892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.072097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41640a68-a195-49e9-9841-d12b573a0977 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.121062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.121695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.121976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.122244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.122952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:52.123255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:52.126341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:52.137890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 171/340] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:52.158975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.214657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de0bc-8c5a-42a6-a29e-c01d7914fb70 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.283226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.283787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.284062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.284331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.305192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e03fe9ea-7691-4079-b7eb-01b1cc265166 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:52.311440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:52.313525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:52.315503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 170/341] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:52.319648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fc2c-90c8-4339-a119-2924eb07e1f5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.319924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fc2c-90c8-4339-a119-2924eb07e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.320195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7fc2c-90c8-4339-a119-2924eb07e1f5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.322625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 171/342] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:52.324921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7fc2c-90c8-4339-a119-2924eb07e1f5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.345859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:52.356473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0eb3-9009-4138-bc8a-6ef67b26cde7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.396194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:52.436538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.437159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.437437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.437701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.516908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7fc2c-90c8-4339-a119-2924eb07e1f5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.567062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.567963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.568390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.568810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.569724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:52.570265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:52.571824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518205aa-6a05-45be-8110-56bb69147aa0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:52.579163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:52.580823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:52.585093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 172/343] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:52.586553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.587066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.587536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.619557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:52.621744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:52.692938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 172/344] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:39:52.694224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.714165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:52.719195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:52.735974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 173/345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:52.736858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:52.853829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4ef77-160d-48e1-bb2e-66e0c3eaded5 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.897320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.898314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.898842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.899329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.928938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87157f-9f33-4641-8b0b-d3316a0a3e75 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:52.984318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:52.985491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:52.985935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:52.986357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:52.987801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:52.989613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:52.991421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 174/346] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:52.998044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.012147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.013978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7590d68-72b8-46ee-b1ec-b2a790aeb9fc None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.056514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.134359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.204694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1fc9-9b77-4afb-96f0-353e40bc2acf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.212474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 173/347] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:53.214932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d5907-3b6b-43cf-843b-990c6234ae34 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.215227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d5907-3b6b-43cf-843b-990c6234ae34 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.215504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d5907-3b6b-43cf-843b-990c6234ae34 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.217263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 174/348] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:53.222091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d5907-3b6b-43cf-843b-990c6234ae34 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.268830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.270189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.270696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.271185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.282537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.284398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685897c-189f-467b-a58d-d8223889c4af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.286792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:52 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:53.293627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.294207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.294709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.321764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:53.324011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:53.356549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d5907-3b6b-43cf-843b-990c6234ae34 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.376128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.397549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.398677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.399199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.399685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.401644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.403613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.405617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 175/350] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:53.406811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.419824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 176/351] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:53.427552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.520108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe894-a4a1-494a-9504-c783d48472f8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.530983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.558864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.559936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.560416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.560888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.561819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:53.562330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.564045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.570225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 176/352] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=7c78b3797aea4e3280d32765c45d23df => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:53.571313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.580381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f618163-ff4d-47c1-8ac4-b0854f06c358 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.625667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.626376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.626629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.626907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.628935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.639619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f4c9d3e8-96f5-4260-9c4b-e7959bd3e0da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:53.644596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:53.645529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:53.646362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:53.651640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d4fd-bd84-4cf3-805c-d840e09c4958 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.651640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d4fd-bd84-4cf3-805c-d840e09c4958 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.651640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d4fd-bd84-4cf3-805c-d840e09c4958 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.651640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 178/354] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:53.654705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.654991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.655265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.680469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:53.681734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:53.711640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9679418-f66c-42e5-bde8-4e0b12bcef63 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.752910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.754039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.754543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.755134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.756611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.758314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.761101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 177/355] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:53.761521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 179/356] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:53.762476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.767569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.849422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.885044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f062b-19bd-4b80-9750-f4ad414ebb70 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.904962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:53.923208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c00fa8-2e0b-465a-9607-951c76fcc282 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:53.931423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.932936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.933489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.934095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.935068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:53.935588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.937289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.947300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 178/357] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:53.954285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:53.970738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:53.971125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:53.971459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:53.971733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:53.972475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:53.972791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:53.974231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:53.981619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:54.012850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.086569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eee04-fd53-4fce-aee5-9745a7a8d232 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.129569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.130621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.131113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.131605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.187721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.217639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:54.219972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:54.220641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0faf-5153-48f6-8318-7254f11745f2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.236566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.264168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.264819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.265107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.265378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.278708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4d45d7b7-231d-4325-9e16-eeadc50392d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:54.284290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:54.285182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:54.285983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 181/359] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:54.287143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6e1-a670-4fd4-b324-e1165bace759 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.287143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6e1-a670-4fd4-b324-e1165bace759 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.287297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb6e1-a670-4fd4-b324-e1165bace759 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.288754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 182/360] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:54.289447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cb6e1-a670-4fd4-b324-e1165bace759 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.342933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 179/361] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:39:54.343792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.438579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cb6e1-a670-4fd4-b324-e1165bace759 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.457638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.491759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e19f-cbb7-400b-bf18-97b7a0a4899c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.497935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.497935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.497935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.497935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.513304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-64ecbcc5-5b6b-46d2-b06b-739ab42b4a4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:54.518343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:54.519370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:54.520187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:54.521282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 183/362] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:54.522439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.522649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.522889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.542003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.543012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.543489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.543972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.544859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:54.545360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:54.545565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:54.546588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:54.547452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:54.560012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 180/363] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:54.561479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.577169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.622876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 184/364] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:54.630538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.720256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3685-9174-4caa-bc00-032df5868d11 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.743075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c2e8-166c-478d-8c20-709f128ec91b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.783459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.784078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.784306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.784440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.784622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.785337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:54.785643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:54.785872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.786352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.786842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.787134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:54.791215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:54.793554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:54.795515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 181/365] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:54.796426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 185/366] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:54.796752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.801983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:54.817280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.839537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:54.918565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80134e95-a1b8-4847-bfe9-3f219fed1682 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:54.960462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:54.961104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:54.961382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:54.961657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:54.964538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:54.965941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:54.969881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 186/367] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:54.980386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.048884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddfc23-62ac-4783-baf4-4740f30d1978 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.109288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.109987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.110269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.113157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] PATH_INFO: `/v3/users/ee4f8e5b76924164a6224bf55b92b41d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.113157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] RBAC: Authorizing `identity:get_user(user_id=ee4f8e5b76924164a6224bf55b92b41d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.115564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.117341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 182/368] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/users/ee4f8e5b76924164a6224bf55b92b41d => generated 293 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:55.118126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.128844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.137668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc1fa0-7d7f-4de7-a598-b842a46f18b5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.177210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.178230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.178849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.179247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.216025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.248764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cea81-d48f-4ef6-8826-e3ae3413d3ae None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.274886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.275254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.287157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.287767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.288046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.288311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.289037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=17e6c4470b9442d68be311b25262e42d) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:55.289340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.291142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.297599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 183/369] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=7c78b3797aea4e3280d32765c45d23df => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:55.298408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.387201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.398068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 187/370] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:55.398795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.431872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8689108-fe8f-4926-8eeb-909391c15e91 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.471846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.472485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.472817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.473145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.483624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.489549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:55.496604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:55.497504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:55.498301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 184/371] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:55.501476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.540192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f370d-98be-4ae9-b531-01d54f662db6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.583012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.583901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.584249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.584588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.585427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] RBAC: Adding query filter params (role.id=0b8995b146864249aeeb7d42c6f75d7e, user.id=ee4f8e5b76924164a6224bf55b92b41d, scope.system=all) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:55.585743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.587429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.602916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 188/372] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/role_assignments?user.id=ee4f8e5b76924164a6224bf55b92b41d&scope.system=all&role.id=0b8995b146864249aeeb7d42c6f75d7e => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:55.604604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.655193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c5cce-7588-4f9e-92fd-888b986b121d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.714052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.748017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.748695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.748999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.749259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.750430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.752071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.753916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 185/373] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:55.759337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.784186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdc5d9-ab4d-412e-b9a5-a2e7908cbee8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:55.862375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:55.864956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:55.865721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:55.867603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:55.868495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:55.869062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:55.873446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:55.889837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 189/374] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:55.895661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:55.906886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:55.981437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd120-6540-4b65-9084-a5fb5e1956ed None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:56.039193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02eafc-41e7-4bc0-8a43-494426a60693 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:56.055144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:56.056271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.057955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.058408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.060976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:56.064096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:56.064548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:56.067648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 186/375] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:56.074923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:56.110410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:56.111516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.112937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.113486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.253669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:56.255473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:56.256003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c59c35-07ed-4923-b82b-fe2be57c2dc1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:56.327235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:56.327778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.328061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.328336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.329078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=17e6c4470b9442d68be311b25262e42d) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:56.329391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:56.331091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:56.339645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:56.343416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 187/376] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:39:56 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:56.412033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 190/377] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:39:56.419326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:56.552456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12230d39-2faf-4637-a24b-5e1ede672e1e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:56.590974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:56.591652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.591937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.592218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] PATH_INFO: `/v3/projects/1375001928324c1f9d2a4c3ccf24704e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.593523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] RBAC: Authorizing `identity:get_project(project_id=1375001928324c1f9d2a4c3ccf24704e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:56.595077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b59557-f117-4220-a18d-9e69f57c27f0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:56.597257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 188/378] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:56 2022] GET /identity/v3/projects/1375001928324c1f9d2a4c3ccf24704e => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:56.602498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:56.623992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:56.740289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bc838-807f-4d44-a211-0154b1504903 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.740629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bc838-807f-4d44-a211-0154b1504903 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.740847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bc838-807f-4d44-a211-0154b1504903 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.742282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 189/379] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:56.744099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471491c-9ba5-4aa6-9cff-b715d5333ac3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:56.755005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca12c-b53f-442a-8fc1-47e1e63ba323 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.755294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca12c-b53f-442a-8fc1-47e1e63ba323 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.755572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca12c-b53f-442a-8fc1-47e1e63ba323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.791667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-42fca12c-b53f-442a-8fc1-47e1e63ba323 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:56.792945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-42fca12c-b53f-442a-8fc1-47e1e63ba323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:56.799177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:56.799836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:56.800116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:56.800391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] PATH_INFO: `/v3/groups/779622fd9c7f43eca211e630eea8259b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:56.812302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] RBAC: Authorizing `identity:get_group(group_id=779622fd9c7f43eca211e630eea8259b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:56.814555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:56.816732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 191/380] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:56 2022] GET /identity/v3/groups/779622fd9c7f43eca211e630eea8259b => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:56.822687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:56.898802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:56.906968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 190/381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:56.949429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:56.969589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6946934-9b48-45de-89c0-de82495a4128 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.005834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:57.006448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.006757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.007031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.007759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] RBAC: Adding query filter params (group.id=779622fd9c7f43eca211e630eea8259b, role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=1375001928324c1f9d2a4c3ccf24704e) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:57.008066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:57.010958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6fb91-9c2c-40bc-86fd-45a37f7f15e3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:57.016764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 192/382] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:39:56 2022] GET /identity/v3/role_assignments?group.id=779622fd9c7f43eca211e630eea8259b&scope.project.id=1375001928324c1f9d2a4c3ccf24704e&role.id=ada491db6331406099f2a486b26a3cc4 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:57.063854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f578-d8d0-4c44-bcba-105f2d1e9075 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.064189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f578-d8d0-4c44-bcba-105f2d1e9075 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.064398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f578-d8d0-4c44-bcba-105f2d1e9075 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.065975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 191/383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:57.071737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.072273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.073414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.098783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:57.100338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:57.188622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 193/384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:57.272787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5f578-d8d0-4c44-bcba-105f2d1e9075 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.415764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5f578-d8d0-4c44-bcba-105f2d1e9075 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.464665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:57.465233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.465511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.465780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.484758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:57.488968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:57.489650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:57.490553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:57.491351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 192/385] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:57.497066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.578580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:57.589239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:57.625362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75a1e0-5608-476c-9182-927878f07ad4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.633128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d208-6a5b-4d7d-9ee3-687368a300d4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.673078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:57.673965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.674350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.674954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.681362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:57.682990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.683454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.684051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.685087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:57.685582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:57.687350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:57.689100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:57.695404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:57.696357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:39:57.697150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 194/386] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:39:57.700883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 193/387] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:57.706548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.707008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.825307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45866-ff45-4e15-bd46-e7dec095fd51 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.851774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:57.870321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:57.871091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:57.871374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:57.871648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:57.873605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:57.879172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:57.881286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 194/388] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:57.888666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:57.936005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421c91-3b36-4cb0-8168-1e3539c40757 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:57.997783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:58.007083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793001-6160-40af-8629-a851b783d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.007709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793001-6160-40af-8629-a851b783d762 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.007994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793001-6160-40af-8629-a851b783d762 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.008274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793001-6160-40af-8629-a851b783d762 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.008959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793001-6160-40af-8629-a851b783d762 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:58.009269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793001-6160-40af-8629-a851b783d762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.012801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793001-6160-40af-8629-a851b783d762 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.031655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 195/389] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:58.056139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793001-6160-40af-8629-a851b783d762 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:58.082406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3103159-5ed5-490e-9984-990f624232d7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:58.139850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.140492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.140779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.141057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.142196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.143707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.145520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 195/390] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:57 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:58.151743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:58.216441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:58.222477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793001-6160-40af-8629-a851b783d762 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:58.265576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff4c7-5576-43ea-a184-92dfb450c95c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:58.304650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.305383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.305595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.305877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/groups/bd44c81286db4318bfa82996b22c2737/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.313254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, group_id=bd44c81286db4318bfa82996b22c2737, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.314784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128a2ef-a40f-46f0-9057-0d1c4497569c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.325300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.325979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.326263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.326567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.327830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.329314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.332347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 196/391] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:58 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:58.339363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:58.349029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 196/392] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 12 11:39:58 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/groups/bd44c81286db4318bfa82996b22c2737/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:39:58.440075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:58.458685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62986cf3-a533-4b85-92e8-485418c6b9a2 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.458993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62986cf3-a533-4b85-92e8-485418c6b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.459253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62986cf3-a533-4b85-92e8-485418c6b9a2 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.460760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 197/393] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:39:58.467198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.467482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.467759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.496814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:58.498043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:58.525143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131512-d760-40ba-ade4-cecd654506c3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:58.596997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.597650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.597933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.598211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.600596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.602061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.604171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 197/394] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:58 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:39:58.609934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:58.613496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 198/395] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:39:58.739949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d8829-2473-4ea2-a960-357836c3695f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:58.771153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:58.778017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:58.778728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:58.779014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:58.779294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/groups/bd44c81286db4318bfa82996b22c2737/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:58.781641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d5c798010c4e5b8344a78e2b4e8f24, group_id=bd44c81286db4318bfa82996b22c2737, role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:58.783122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:58.802312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 198/396] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 12 11:39:58 2022] PUT /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24/groups/bd44c81286db4318bfa82996b22c2737/roles/b42a9d0451fc419ab1547c88f1248317 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:39:58.955843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:59.008898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:59.075436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15007e94-dd3d-4c69-bcdc-aee2d441e1d5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:59.111045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:59.112018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.112505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.112984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.124869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:39:59.129800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:39:59.130746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:39:59.131617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 199/397] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:39:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:39:59.137490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:59.293123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846dbdf-e092-4f34-a15b-cf41ff7896c4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:59.334330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:59.335007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.335251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.335521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.336190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:39:59.336488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:59.338675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1e2ed-d769-4499-ac21-389e7dec7e76 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:59.350078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 199/398] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:39:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:59.355822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:59.433279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3453a-2573-42e4-9f8a-74dc193c0965 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.433632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3453a-2573-42e4-9f8a-74dc193c0965 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.433829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3453a-2573-42e4-9f8a-74dc193c0965 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.435228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 200/399] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:39:59.441570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca91639-13ba-409e-b126-58058782a87a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.441859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca91639-13ba-409e-b126-58058782a87a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.442135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca91639-13ba-409e-b126-58058782a87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.480058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dca91639-13ba-409e-b126-58058782a87a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:39:59.481263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dca91639-13ba-409e-b126-58058782a87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:39:59.514210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83878754-0f4e-4aab-822e-16a134795652 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:59.522463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:59.558628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:59.559218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.559944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.560226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.561723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:59.563221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:59.569899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 200/400] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:39:59 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:59.570865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:59.605696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:39:59.643756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 201/401] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:39:59.727578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcd159-46bb-442c-a6b2-916736415cb7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:39:59.782144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:39:59.782830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:39:59.783110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:39:59.783378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:39:59.784537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:39:59.785983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:39:59.787960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 201/402] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:39:59 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:39:59.794314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca91639-13ba-409e-b126-58058782a87a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:39:59.960714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca91639-13ba-409e-b126-58058782a87a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:00.035254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:00.035936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.036211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.036482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/groups/bd44c81286db4318bfa82996b22c2737/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.043924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, group_id=bd44c81286db4318bfa82996b22c2737, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:00.045442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a7fd-943a-44d8-8d5d-4293c235b33d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:00.065065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 202/403] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 12 11:39:59 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/groups/bd44c81286db4318bfa82996b22c2737/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:00.172341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:00.227770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:00.305786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44318342-a127-4576-b8df-410fc8c79542 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:00.343317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:00.344377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.344858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.345328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.358407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:00.364159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:00.365040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:00.366365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:00.367424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112fb6-d365-4426-9386-1a73e81cab48 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.367718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112fb6-d365-4426-9386-1a73e81cab48 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.367996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92112fb6-d365-4426-9386-1a73e81cab48 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.371734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 203/405] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:00.371973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 202/405] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:00.375926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:00.376285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.376575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.376854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.399762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:00.404971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:00.462295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:00.509181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 204/406] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:00.520515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5879b4-b6a1-43c0-a2e9-84a3a873f79a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:00.522395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:00.584314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:00.584930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.585211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.585482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.586154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:00.586454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:00.588164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:00.602285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 203/407] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:00.608097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:00.738153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f04d61-1ad6-4f88-8a00-260e894b7361 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:00.780453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:00.781316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:00.781737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:00.782148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:00.783833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:00.786122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:00.788956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 205/408] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:00 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:00.794698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:00.872729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:00.934896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:00.981134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11dbb4-9280-4c6f-8b15-0a2b5bcd0e41 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.008279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bdea4-b0e7-4afc-91aa-b6dfd4a72629 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.046992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.047550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.047826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.048095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.050164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:01.051616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:01.054020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 204/409] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:00 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:01.060722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.071642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.072448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.072812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.073069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.092266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:01.099851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:01.100734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:01.101797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 206/410] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:01.108860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.116635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:01.195181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2793-ae48-45a0-94bc-2f614d7b5150 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.234247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.234954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.235235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.235505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d/groups/bd44c81286db4318bfa82996b22c2737/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.237663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] RBAC: Authorizing `identity:create_grant(project_id=17e6c4470b9442d68be311b25262e42d, group_id=bd44c81286db4318bfa82996b22c2737, role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:01.239154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06cb97-4711-4a4c-9571-55cbad19ee2c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:01.255081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 205/411] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 12 11:40:01 2022] PUT /identity/v3/projects/17e6c4470b9442d68be311b25262e42d/groups/bd44c81286db4318bfa82996b22c2737/roles/b42a9d0451fc419ab1547c88f1248317 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:01.366502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:01.371027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1c4ad-603d-469f-bfc3-5c22ba6e3696 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.403072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328868e-85e9-44f2-b91f-ffe937a3bf7f None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.403972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328868e-85e9-44f2-b91f-ffe937a3bf7f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.404422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328868e-85e9-44f2-b91f-ffe937a3bf7f None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.405973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 206/412] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:01.411458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.412033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.412481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.429014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.429650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.429935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.430208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.430926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:01.431234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:01.433556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:01.434636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:01.436187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:01.449146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 207/413] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:01.456623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.521753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 207/414] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:01.579414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd57c-afe8-4dcb-bb9a-9501424eac4b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.633419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.635471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.636022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.636492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] PATH_INFO: `/v3/groups/779622fd9c7f43eca211e630eea8259b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.642872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] RBAC: Authorizing `identity:get_group(group_id=779622fd9c7f43eca211e630eea8259b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:01.644558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce2252-f87e-472a-82df-194aefd67682 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:01.646369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 208/415] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:01 2022] GET /identity/v3/groups/779622fd9c7f43eca211e630eea8259b => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:01.652498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.692270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:01.726389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc899f-7ba6-4ce1-9671-b85eb9df71de None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.726958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc899f-7ba6-4ce1-9671-b85eb9df71de None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.727381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc899f-7ba6-4ce1-9671-b85eb9df71de None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.729400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 209/416] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:01.734681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.734966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.735244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.775771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:01.776986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:01.784976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a768bd-903b-4fd3-a6b7-b3689bcafc01 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:01.821687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:01.823083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:01.823604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:01.824050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] PATH_INFO: `/v3/projects/1375001928324c1f9d2a4c3ccf24704e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:01.825449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] RBAC: Authorizing `identity:get_project(project_id=1375001928324c1f9d2a4c3ccf24704e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:01.827210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:01.829390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 208/417] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:01 2022] GET /identity/v3/projects/1375001928324c1f9d2a4c3ccf24704e => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:01.835257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:01.836756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.910709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 210/418] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:01.911670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:01.946839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:01.958998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c724ef-1309-494c-bd2a-f712cd90bf37 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.004151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.004808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.005083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.005349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] PATH_INFO: `/v3/projects/1375001928324c1f9d2a4c3ccf24704e/groups/779622fd9c7f43eca211e630eea8259b/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.010775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] RBAC: Authorizing `identity:create_grant(project_id=1375001928324c1f9d2a4c3ccf24704e, group_id=779622fd9c7f43eca211e630eea8259b, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:02.012165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:02.036937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 209/419] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Sep 12 11:40:01 2022] PUT /identity/v3/projects/1375001928324c1f9d2a4c3ccf24704e/groups/779622fd9c7f43eca211e630eea8259b/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:02.052807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3c6c4-bd49-47cb-a6f5-df42c1f81ac9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.091849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.092503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.092792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.093070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.105773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:02.111095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:02.112091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:02.112901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 211/420] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:02.118646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.264626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bc36d-c94c-466b-ab70-bfd8976fbc92 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.299276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.317648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.318280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.318801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.319072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.320269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] RBAC: Adding query filter params (name=member) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:02.320569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:02.322341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:02.328622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:02.334936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 210/421] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:02.342062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.361717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:02.437552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1befb-3ac4-4f82-ac61-75b3ef71d964 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.472629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d733-e818-4744-9aa7-c4a39f1c0540 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.477140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.477140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.477413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.477485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.489431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:02.494824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:02.495718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:02.496512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:02.500894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 212/422] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:02.502084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.522731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.525083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.525697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.526181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.663640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:02.665133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:02.670887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3bd4-1bc3-4bd6-814e-15e7bf66fcdf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.679323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:02.696006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:02.735204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.736199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.736681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.737147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.738008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:02.742643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:02.744337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:02.758742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 213/423] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:02.764075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.864045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 211/424] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:02.871871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:02.933572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d4f64-c339-4478-afee-3eb512479fca None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.979145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632db989-c20f-453f-8737-e92464176e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:02.980658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db989-c20f-453f-8737-e92464176e7f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:02.981082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db989-c20f-453f-8737-e92464176e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:02.981520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db989-c20f-453f-8737-e92464176e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:02.982725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943bafb-6f1d-42ea-a428-4441748ff81d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:02.997536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.002747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.021583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.022360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.022769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.023242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.025019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.027214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d577af-e0d2-4e2d-bbc7-57e4e2bd3728 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.034451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 212/425] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:03.034451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cc34-a08c-448f-b927-e667dcb07395 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.034451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cc34-a08c-448f-b927-e667dcb07395 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.034451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cc34-a08c-448f-b927-e667dcb07395 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.037688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 213/426] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:03.039133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1cc34-a08c-448f-b927-e667dcb07395 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.075203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632db989-c20f-453f-8737-e92464176e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.077821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632db989-c20f-453f-8737-e92464176e7f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.182999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1cc34-a08c-448f-b927-e667dcb07395 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.217653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 214/427] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:40:03.219311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.219795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.220227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.234794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b162a9-b086-4574-8340-297321dda190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.235548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b162a9-b086-4574-8340-297321dda190 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.236508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b162a9-b086-4574-8340-297321dda190 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.236861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b162a9-b086-4574-8340-297321dda190 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.238191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b162a9-b086-4574-8340-297321dda190 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.240172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b162a9-b086-4574-8340-297321dda190 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.242554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 214/428] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:03.243162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b162a9-b086-4574-8340-297321dda190 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.247696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:03.248886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:03.286147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.304772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.327345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 215/429] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:03.327345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.390471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b162a9-b086-4574-8340-297321dda190 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.463648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.464854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.465956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.466623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] PATH_INFO: `/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.469592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d5c798010c4e5b8344a78e2b4e8f24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.471460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.474154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 215/430] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/projects/e0d5c798010c4e5b8344a78e2b4e8f24 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:03.479927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9ff4d-0d2b-45e5-a9b4-5b064b358686 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.481029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.575773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.576400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.576678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.576949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.577684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:03.577989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.579467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.593870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 216/431] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=7c78b3797aea4e3280d32765c45d23df => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:03.605857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fd727-6e4b-4522-8122-a86983d55d97 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.629538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.672140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.678802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.679459 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.679744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.680138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.682234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.684079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.691666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 216/432] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:03.693714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.759870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:03.867907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6c3c-c7bd-4f4a-87b8-a8f32b2dc01c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.881633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571aab-4aa9-4613-8668-f0b4f5eac18b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:03.919578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.920640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.921070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.921354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.922113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:03.922812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.923053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.923315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.924035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:03.924340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:03.925776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f97ef-3c3f-4caf-9466-e3292dd5a16e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:03.933452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 217/433] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=e0d5c798010c4e5b8344a78e2b4e8f24&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:03.934485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:03.935231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d53-cd0b-4d5e-8953-eebe0387a426 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.935512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d53-cd0b-4d5e-8953-eebe0387a426 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.935782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d53-cd0b-4d5e-8953-eebe0387a426 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.937046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 218/434] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:03.944229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:03.944603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:03.945668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 11:40:03.946759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:03.948127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:03.948208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 217/435] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:03.948293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:03.958032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:03.972659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:03.974063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:04.024011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.051835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 219/436] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:04.073238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.080142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644fdc4-2b38-4f8d-82ee-3efb0bd40a2f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.133269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.133906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.134185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.134456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.135736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] RBAC: Adding query filter params (name=member) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:04.136225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:04.137619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:04.147586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 218/437] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:40:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:04.153507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.272541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02119e90-fce2-443f-a85e-28bfd573adcc None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.310376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.311032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.311308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.311577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.339219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.364965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.392807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:04.394626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:04.471685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dec26-f6a3-4020-bb91-d290b000f039 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.520616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.521509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.521792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.522392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.526839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 220/438] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:04.533026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.535761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:04.543481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:04.544426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:04.546328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 219/439] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:04.550723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.579211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.625146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.647820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0f1c-f1f9-41c0-814a-04b2bbb8389b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.670807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed6d28-4e1e-4913-80e0-9d1a88becadb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.682908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.683538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.683814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.684084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.685200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:04.686690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e7944-d911-4d21-abdc-e2c3b7703eef None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:04.688299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 221/440] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:04.689403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f13c-33df-4afe-bdc6-d13a0691e2a0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.689681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f13c-33df-4afe-bdc6-d13a0691e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.689956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f13c-33df-4afe-bdc6-d13a0691e2a0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.691310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 222/441] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:04.693768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773f13c-33df-4afe-bdc6-d13a0691e2a0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.707908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.708655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.708931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.709196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.709887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:04.710185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:04.711577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4daec-30ae-4e4a-a708-0c86f173393f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:04.719650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 220/442] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:04.720771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.721050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.721322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.738880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:04.740065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:04.799794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773f13c-33df-4afe-bdc6-d13a0691e2a0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.804456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 221/443] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:04.805267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.826087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.840705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.841433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.841763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.842101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.843309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:04.844735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:04.846465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 223/444] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:04.852107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:04.868639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:04.922918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08ef63-8835-4bca-bac5-196077935053 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:04.956900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:04.957504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:04.957787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:04.958062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:04.966036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b4679-d5ef-4c47-b780-b2b00a5e4cd9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.003153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.003990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.004417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.004931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.005907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=7c78b3797aea4e3280d32765c45d23df, scope.project.id=17e6c4470b9442d68be311b25262e42d) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:05.006213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.007669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.024009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 224/445] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=7c78b3797aea4e3280d32765c45d23df => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:05.040478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.042182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.055343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.078415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.157848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 222/446] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:05.164156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.177787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.206395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067188a5-5ca2-46f8-bb4c-aa88ea00d3ea None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.247976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.248607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.248881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.249149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.261336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:05.267670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:05.268578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:05.269420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 225/447] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:05.274824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.282347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42be0-f2fe-4afe-b91c-347c7a42718a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.294349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.326178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.326818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.327097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.327365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] PATH_INFO: `/v3/projects/17e6c4470b9442d68be311b25262e42d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.328528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] RBAC: Authorizing `identity:get_project(project_id=17e6c4470b9442d68be311b25262e42d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.329961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.332072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 223/448] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/projects/17e6c4470b9442d68be311b25262e42d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:05.337083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.398807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefdedd-17ee-4729-9ec2-5b1867f90769 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.436941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.437547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.437835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.438108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.438800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:05.439105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.440485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.445411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.450429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 226/449] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:05.455886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.464660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a265a0-6f4e-42ea-96ce-979384212e83 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.498274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427915-feda-4980-8071-04f568f60b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.498934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427915-feda-4980-8071-04f568f60b75 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.499219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427915-feda-4980-8071-04f568f60b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.499497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427915-feda-4980-8071-04f568f60b75 None admin] PATH_INFO: `/v3/groups/bd44c81286db4318bfa82996b22c2737` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.500624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17427915-feda-4980-8071-04f568f60b75 None admin] RBAC: Authorizing `identity:get_group(group_id=bd44c81286db4318bfa82996b22c2737)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.502121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17427915-feda-4980-8071-04f568f60b75 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.503731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 224/450] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/groups/bd44c81286db4318bfa82996b22c2737 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:05.509996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427915-feda-4980-8071-04f568f60b75 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.554429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b0162-5e43-442f-a29d-42e9429a0e89 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.556179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.589378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.589898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.590175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.590447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.608852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427915-feda-4980-8071-04f568f60b75 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.641745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.642332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.642640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.642914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.643637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] RBAC: Adding query filter params (group.id=bd44c81286db4318bfa82996b22c2737, role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=17e6c4470b9442d68be311b25262e42d) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:05.643947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.645370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.651226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 225/451] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/role_assignments?group.id=bd44c81286db4318bfa82996b22c2737&scope.project.id=17e6c4470b9442d68be311b25262e42d&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:05.654411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.657398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.659229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.765104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 227/452] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 11:40:05.770874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:05.872838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cd900-de96-4cbd-a4ba-95b5eb51b1be None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:05.883174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:05.908316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:05.908951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:05.909231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:05.909503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] PATH_INFO: `/v3/projects/1375001928324c1f9d2a4c3ccf24704e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:05.910663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] RBAC: Authorizing `identity:get_project(project_id=1375001928324c1f9d2a4c3ccf24704e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:05.912113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:05.913731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 226/453] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/projects/1375001928324c1f9d2a4c3ccf24704e => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:05.920081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:06.018239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:06.021635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7306-f1ae-4427-aaa6-7549a9276f2b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:06.055525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:06.056142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:06.056457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:06.056728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] PATH_INFO: `/v3/groups/779622fd9c7f43eca211e630eea8259b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:06.057819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] RBAC: Authorizing `identity:get_group(group_id=779622fd9c7f43eca211e630eea8259b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:06.059295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45993477-d566-4ebe-b07d-2b45af2e3699 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:06.060833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 228/454] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Sep 12 11:40:05 2022] GET /identity/v3/groups/779622fd9c7f43eca211e630eea8259b => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:06.066734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:06.167918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792e251-a8e6-4043-8c3c-7b8add8154cf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:06.200629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:06.201226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:06.201509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:06.201785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:06.202538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] RBAC: Adding query filter params (group.id=779622fd9c7f43eca211e630eea8259b, role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=1375001928324c1f9d2a4c3ccf24704e) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:06.202853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:06.204314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769992-ad13-4c25-ad67-70d53851a953 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:06.209924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 227/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Sep 12 11:40:06 2022] GET /identity/v3/role_assignments?group.id=779622fd9c7f43eca211e630eea8259b&scope.project.id=1375001928324c1f9d2a4c3ccf24704e&role.id=ada491db6331406099f2a486b26a3cc4 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:08.835141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7a2e8-559c-4eec-be8e-774bb4b27546 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:08.836191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7a2e8-559c-4eec-be8e-774bb4b27546 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:08.836696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7a2e8-559c-4eec-be8e-774bb4b27546 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:08.839065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 229/456] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:08.844647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f334e97-70c2-4d90-8795-2bb58251272c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:08.845485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f334e97-70c2-4d90-8795-2bb58251272c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:08.845960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f334e97-70c2-4d90-8795-2bb58251272c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:08.871527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5f334e97-70c2-4d90-8795-2bb58251272c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:08.872749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5f334e97-70c2-4d90-8795-2bb58251272c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:08.919735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:08.956559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 228/457] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:09.144395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e681d33-5bbd-4219-aaa7-2fea1060b977 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.145044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e681d33-5bbd-4219-aaa7-2fea1060b977 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.145511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e681d33-5bbd-4219-aaa7-2fea1060b977 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.147303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 230/458] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:09.160182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.160472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.160756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.182825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:09.184343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:09.192822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a686d7-c0d8-4c60-9dec-3d8287cff003 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.193395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a686d7-c0d8-4c60-9dec-3d8287cff003 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.193901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a686d7-c0d8-4c60-9dec-3d8287cff003 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.195467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 231/459] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:09.201848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.202375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.202862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.235498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:09.237121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:09.264336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 229/460] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:09.353613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 232/461] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:09.412897 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:09.525812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3178ce-c4d7-49e6-8531-4f7d747c20fb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:09.527454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:09.562170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:09.562834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.563118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.563393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.574605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:09.576051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:09.600769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:09.601751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:09.602784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:09.602784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:09.602784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:09.602784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 230/462] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:09.609974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:09.632973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:09.749712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8f226-16f0-4aea-b37e-d76feaf89035 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:09.757590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:09.761056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91455694-1fef-4e82-bd6b-fb65fa2b2af1 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:09.788206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:09.788888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.789172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.789448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.798033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:09.799062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.799351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.799636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.800282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:09.800585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:09.800682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:09.801966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:09.802069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:09.823923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 233/463] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e6369797-7597-4b61-b7f4-f5a070d8fac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:09.824182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:09.825225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:09.826322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:09.826322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:09.826322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:09.826537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 231/464] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9686-b517-4c7f-b98f-3e6d46fc1954 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9686-b517-4c7f-b98f-3e6d46fc1954 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9686-b517-4c7f-b98f-3e6d46fc1954 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 232/465] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:09.826621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9686-b517-4c7f-b98f-3e6d46fc1954 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:09.838542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:09.930871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9b4ce-ff92-4b33-ad8e-04a3dd902990 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:09.940571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9686-b517-4c7f-b98f-3e6d46fc1954 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:09.968998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:09.969738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.969960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.970236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.981662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:09.982423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:09.982749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:09.982861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:09.983024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:09.983672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:09.983975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:09.984220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:09.985320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:09.998054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 233/466] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:09.999399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c93c2e09-3fb3-4b69-84a6-0d2de0d44303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:10.001080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:10.002281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:10.003153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:10.003153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:10.003153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:10.009374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 234/467] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:10.011123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.011552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.011969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.030967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:10.032364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:10.035962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.069294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.107717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450a132-1c4a-4ea2-bc07-186a6f509c14 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.122792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 235/468] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:10.123527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.152703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.154378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.155031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.155614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.156726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.159175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.198762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 234/469] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:09 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:10.198762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.287879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a050-a1e2-411e-92a9-de4aecb7ed68 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.379887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d65e0-d77b-4605-8392-5e1197831a77 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.399282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.400462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.401695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.401899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.402572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.404059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.409337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.419332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.426416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.427252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.427553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.427804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.428597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:10.428911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.430259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.434382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 236/470] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:10 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:10.436083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.445360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 235/471] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:10.451368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.579732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a600-840b-4fb3-91fa-999959ab8b99 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.593836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786b7a7-fe49-47c6-a1bc-f201e1874e0a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.624369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.625006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.625284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.625554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.631748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.632359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.632638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.632909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.633565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.635567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5406001-591f-44a0-8978-ab01b5e74285 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.637568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.639117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-6d3baaeb-90bd-45ee-a5f0-a061f018dc9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:10.656451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:10.657544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:10.658409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:10.658409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:10.658409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:10.658409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 237/472] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:10.659182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fa6a-0640-4b94-b393-00541442cba0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.659182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fa6a-0640-4b94-b393-00541442cba0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.659468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fa6a-0640-4b94-b393-00541442cba0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.660916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 238/473] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:10.667175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3fa6a-0640-4b94-b393-00541442cba0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:10.667363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 236/474] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:10 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:10.668733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.669022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.669297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.689590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:10.690841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:10.731237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.748531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:10.790057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 237/475] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:10.820380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3fa6a-0640-4b94-b393-00541442cba0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:10.865374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:10.866059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:10.866410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:10.866717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:10.867413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:10.867719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:10.869263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:10.883134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 239/476] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:10.888819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:11.003271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fb111-047e-4fef-87ba-38e3b65681a0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:11.039403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:11.040021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:11.040336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:11.040607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:11.041269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:11.042726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:11.068865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 238/477] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:10 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:11.083035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:11.199852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973ea1b-0375-4f2c-8fa4-d0e0f623a8f3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:11.211120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:11.242461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:11.243128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:11.243947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:11.244224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:11.244918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:11.246335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:11.257034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.sql.core [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 11:40:11.257034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 11:40:11.257034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [parameters: {'id': '8228acb4f7354b7ca67931d760452259', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 11:40:11.257034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81386) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result.read() Sep 12 11:40:11.273600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return ref.to_dict() Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:40:11.274569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application yield resource Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise self.value Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application session.commit() Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 11:40:11.276695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.session.flush() Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application compat.raise_( Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 11:40:11.277608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:40:11.278481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result.read() Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:40:11.280933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [parameters: {'id': '8228acb4f7354b7ca67931d760452259', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:11.282874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 11:40:11.284585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:11.285638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 240/478] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 12 11:40:11 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:11.285716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:11.314910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:11.397500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0594-686a-4eb1-891e-bd77f17749d0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:11.434990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:11.435628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:11.435937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:11.436181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:11.446915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-36f44a34-6b4d-4382-985f-988723ad64d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:11.452338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:11.454694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:11.456701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 239/479] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:11.460462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:11.565291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125773-93e1-4929-b1c5-3a6009baabd3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:11.600991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:11.601599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:11.601875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:11.602144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:11.603383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:11.603750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:11.605195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91b51-9986-479a-b946-03f06099ae6d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:11.613969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 241/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:13.058836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9554f1-d695-45b8-bd3a-9207d55f8fb2 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.059536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9554f1-d695-45b8-bd3a-9207d55f8fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.059536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9554f1-d695-45b8-bd3a-9207d55f8fb2 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.060953 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 240/481] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:13.066468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d39c-9a94-4736-ac75-43e397ce505b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.067075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d39c-9a94-4736-ac75-43e397ce505b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.067075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435d39c-9a94-4736-ac75-43e397ce505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.095270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a435d39c-9a94-4736-ac75-43e397ce505b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:13.096491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a435d39c-9a94-4736-ac75-43e397ce505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:13.137616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:13.194507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 242/482] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:13.210223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717afe9-faa5-47e8-91ba-7d6455529305 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.210529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717afe9-faa5-47e8-91ba-7d6455529305 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.210814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717afe9-faa5-47e8-91ba-7d6455529305 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.213281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 241/483] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:13.218487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aff94-4b29-43c8-951e-1a53156f6a82 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.218805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aff94-4b29-43c8-951e-1a53156f6a82 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.219081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aff94-4b29-43c8-951e-1a53156f6a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.241095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4c3aff94-4b29-43c8-951e-1a53156f6a82 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:13.242276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4c3aff94-4b29-43c8-951e-1a53156f6a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:13.346076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 243/484] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:13.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717afe9-faa5-47e8-91ba-7d6455529305 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:13.659522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca46bb-0951-4da1-95f6-aa1c6b998abc None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.659835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca46bb-0951-4da1-95f6-aa1c6b998abc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.660063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca46bb-0951-4da1-95f6-aa1c6b998abc None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.661749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 244/485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:13.667005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.667290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.667562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.703027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:13.704274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:13.705496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:13.721147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717afe9-faa5-47e8-91ba-7d6455529305 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:13.757161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:13.757814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.758091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.758361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.769797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b6886323-c896-40b5-a27a-5de4607a5f46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:13.775350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:13.776409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:13.777243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 242/486] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:13.778425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16d08-200f-4f6a-bfda-9e83648368e5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.778425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16d08-200f-4f6a-bfda-9e83648368e5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.778425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16d08-200f-4f6a-bfda-9e83648368e5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.779576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 243/487] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:13.780103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a16d08-200f-4f6a-bfda-9e83648368e5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:13.798654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 245/488] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:13.799959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:13.831958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:13.905419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a16d08-200f-4f6a-bfda-9e83648368e5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:13.921038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a5c86-bae3-418a-9f0b-664c7ff66279 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:13.940662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:13.941584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.942065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.942578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.953843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b628a5af-4f45-48b6-89ed-af675e7b953e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:13.959325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:13.960208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:13.961038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:13.961671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:13.961671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.961671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.961671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.961990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:13.963083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:13.964262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:13.969412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 244/489] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:13.971130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:13.971622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:13.972095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:13.977435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 246/490] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:13.978455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:13.983185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:13.995200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:13.996730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:14.070562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 245/491] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:14.070562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.083693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.104889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af7dfd-7332-4029-b6f7-a4188678140a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.148960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.150419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.150954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.151429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.152292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:14.152791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.154428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.163575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 247/492] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:14.164701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.190783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6454e05-bfdc-4acc-9cb2-b098072350c0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.227456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.228701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.228984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.229278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.231417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.242605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.244110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:14.260431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.261391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:14.262361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:14.262361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.262361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:14.262361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 246/493] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.263023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.277521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ed3a-ba95-4520-84b3-3007c1c1f7a6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.314944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.315928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.316405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.316868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.328241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:14.333888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.334826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:14.335645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:14.343490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 248/494] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.344631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.356677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.377118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4443-5621-429c-8ac0-e2251d728d9a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.412207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.412850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.413216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.413387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.425035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.426433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:14.441708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.442733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:14.443591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:14.443591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.443591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:14.443591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 247/495] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.444422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.463634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7698-2b83-4098-b4bf-9ba7b3c75068 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.488134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.499851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.500727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.501021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.501295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.501944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:14.502246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.504136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.517077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 249/496] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:14.521534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.562629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f225-9b83-4551-9bf6-7c1f7af7d652 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.593166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.613499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.614119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.614563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.614983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.616016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:14.616472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.618727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.629340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 248/497] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:14.630563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.638635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ac11e-e48b-4d55-af57-53a508602970 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.676209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.676764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.677049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.677323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.692254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:14.697590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.698495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:14.699351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 250/498] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.700029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.740637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.743798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afd8fb-5047-4af6-b627-afbb39e4addf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.777812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.778419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.778731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.779009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.779652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:14.779958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.781315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.793155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 249/499] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:14.793830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.803120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b038c5c-c51c-44bc-ae70-cc145809712d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.829975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:14.838864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.839488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.839765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.840034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.847032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.848483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:14.861913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.862899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 251/500] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.864151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.896590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a20621-ed22-4294-b33f-e7d457bd1aed None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.931770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:14.932361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.932637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.933038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.949060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:14.950543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-58fa8aa4-4d54-499d-ba7e-929b82da3a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:14.965482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:14.966395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:14.967284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:14.967284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:14.967284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:14.967284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 250/501] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:14.967937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5e68a-3dd3-4209-a0aa-9a4e0f5f083b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:14.968180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5e68a-3dd3-4209-a0aa-9a4e0f5f083b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:14.968464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5e68a-3dd3-4209-a0aa-9a4e0f5f083b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:14.969972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 251/502] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:14.970706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5e68a-3dd3-4209-a0aa-9a4e0f5f083b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:14.975868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442561-34d6-429c-9e5e-d20767e3736a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:14.985659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.011735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.012329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.012603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.012872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.013946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:15.014245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.015650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.023287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 252/503] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:15.023287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.074096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.078356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5e68a-3dd3-4209-a0aa-9a4e0f5f083b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.114240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.115005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.115171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.115474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] PATH_INFO: `/v3/users/glance` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.123717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.125174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.125361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab917ea0-c193-4b80-8d92-e81802dd0149 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:15.137771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:15.138772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 252/504] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:15.139669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.158747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.159342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.159618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.159886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.160614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:15.160914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.162334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc13f4-c2c6-4a7b-8d6d-4141ed8fd323 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.198304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 253/505] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:15.199325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.199605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.199929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.213007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.217936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:15.219153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:15.244721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40e0f-067b-474e-80ae-13740ced8dab None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.278990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.279611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.279890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.280163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.280802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:15.281112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.282485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.283182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 254/506] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:15.284018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.296832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 253/507] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:15.297738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.324173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.387481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674fad-167c-4c61-ae61-1a87127cb7e2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.409911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980482d-9bcc-4cc7-996b-f3fa46d2b5c5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.422321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.422975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.423260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.423534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.434776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.436235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.443944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.450215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.450791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.451073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.451342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.452070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:15.452373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:15.452549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:15.453366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:15.454244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:15.454244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:15.454244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:15.454244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 255/508] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:15.454981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.458276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.498039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 254/509] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:15.498925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.556818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.567455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f2ec2-79a1-4400-9a58-2221194e4356 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.601289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.601964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455718f-b68b-4a1d-8511-4ae88c546a93 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.602085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.602243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.602875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.636436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.637061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.637345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.637617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.644583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.646036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:15.658151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:15.659207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:15.660349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 255/510] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:15.661054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.671230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.672686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.712941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.759974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.763717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e82e3a-deca-4f17-8cb8-26e0cfdfdbb9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.770453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 256/511] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:15.771287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.801882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.802659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.802941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.803214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.803850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:15.804147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.806016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.817509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 256/512] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:15.818291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:15.868679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5e70-f53a-49f4-9bb5-3d7776f8c959 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.902465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.903106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.903388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.903664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.916949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60ba9c-dd4f-4756-a859-bfd1bd7b50fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:15.928630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.953624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:15.954243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:15.954526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:15.954820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:15.959825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:15.965098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.966484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.971426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:15.972873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:15.982907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:15.984023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:15.985088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:15.985088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:15.985088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:15.986671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 257/513] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:15.987675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.068799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 257/514] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:16.069716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.085127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3edb2-bc1f-4744-adf0-6ed4429c08e6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.119807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.120413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.120690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.120960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.121688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:16.121991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.123511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.140193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 258/515] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 12 11:40:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:16.140985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.159262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.161645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.170982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364644b-6a61-401a-87f8-c951573766d0 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.204397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.205101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.205455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.205790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.216196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.217718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:16.232966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:16.234060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:16.234961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:16.234961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:16.234961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:16.234961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 258/516] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:16.234961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.243647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6254485-6cf2-4258-ac59-086a86f9faeb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.287046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.287672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.287949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.288218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.296004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.297424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:16.309270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:16.310196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:16.311357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 259/517] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:16.312375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.337973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab616a10-d975-407a-bdbd-38859df43e4b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.370716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.371313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.371585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.371858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.372492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:16.372793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.374168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.387051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 259/518] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:16.387839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.391847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.397968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.417353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538eae-db3a-4b29-8680-a484da097352 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.451460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.452080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.452357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.452627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.463252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.464690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:16.479590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:16.480480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:16.481343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:16.481343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:16.481343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:16.481343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 260/519] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:16.481343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.487687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6740-105e-4a45-b71a-bf59d491daba None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.521096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.521701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.521977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.522246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.580729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aebf1-b35a-49f9-a1cf-8c611652f857 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.587667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.589120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.590901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.610872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.617112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.617726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.618552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.618837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.619532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:16.619833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.621210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.633168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 261/520] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:16.634575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.693806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 260/521] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:16.694658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.740534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f5b6-8c90-4923-8c1e-e1c5baaa9267 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.773711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.774534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.774908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.775196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.776002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:16.776318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.777740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.793815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa689-b030-4f32-91f5-a4105f15cc22 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.794773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 262/522] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:16.795352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.795453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.831786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.832380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.832661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.832932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.833567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.835041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.845954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:16.857610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 261/523] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:16 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:16.858424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.910479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9ae7a-f01e-4503-9ae2-0cf87d0207cd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:16.951430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:16.952312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:16.952770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:16.953213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:16.954026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:16.954484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:16.956082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:16.969368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 263/524] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:16.970468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:16.988120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc1302-9693-47e9-826f-ede060bfd4fd None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.029732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.030415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.030725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.030999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.038773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.040896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:17.054460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:17.055501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:17.056725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 262/525] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:17.057274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.078119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.114212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec52ff0-8af1-4599-b191-f6fccd34c53b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.126483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.152157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.152787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.153063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.153335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.164334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.165828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:17.180344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:17.181859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:17.183238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:17.183238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:17.183238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:17.183238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 264/526] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:17.183620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.187022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcc739-10cd-47a4-84b2-b66c65ea632b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.222124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.222762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.223039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.223304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.294401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.295932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.296703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134a6da-a194-4ec5-8c1d-059dc4d2a324 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.316618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.333222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.333854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.334133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.334407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.342090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.343591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:17.357230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:17.358173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 265/527] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:17.359745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.386744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.408697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 263/528] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:17.413685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.475515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3256-b29f-47cf-ab3f-7b887f7cb6ff None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.514828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.515437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.515721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.515992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.516689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:17.516992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.518426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.520126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bac3e-8778-4055-b27c-dfae577e3c89 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.537329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 266/529] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:17.538417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.544171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.562163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.562832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.563110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.563380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.564021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:17.564319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.565685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.578101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 264/530] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:17.583002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.629143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.649742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4570c2-9de8-4753-a9c4-d39f9e7d9d8e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.687268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.688081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.688387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.688666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.689362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:17.689692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.691145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.694852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebae856-3318-4cbe-a94b-7b5475ddb5c7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.703786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 267/531] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:17.704520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.735734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.736485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.736674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.736952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.747943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.749527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:17.764880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:17.765821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:17.766675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:17.766675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:17.766675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:17.766675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 265/532] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:17.766904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.800272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.820863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13340a-50a8-4589-a990-7ab7c2110680 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.856848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.857467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.857748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.858022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.858781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:17.859092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.860534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.875920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ba7f7-c213-446c-b192-10ed80b3cc06 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:17.878713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 268/533] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:17.879798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:17.892380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:17.916618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:17.917292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:17.917572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:17.917842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:17.925471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:17.926999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:17.940321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:17.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 266/534] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:17.947419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.035827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10aa4e-8b80-4f1f-8ef2-02ac22895e33 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.056046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:18.071091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd189e5-94da-476f-9b3e-c0d9920798fd None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.099755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.100670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.100952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.101657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.102930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=02a59e7ff79249eaa7f45cdbe15d0eaa) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.103720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.105848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.109791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.110499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.110881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.111231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.111957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.112328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.113801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.122540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 269/535] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/role_assignments?user.id=02a59e7ff79249eaa7f45cdbe15d0eaa&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:18.122540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.128664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 267/536] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:18.135610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.265561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f4043-248d-4dd9-8e3b-627f8d22c792 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.299405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:18.324432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.325008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.325288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.325558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.335275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.337335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.337870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138c8a7-8c72-48c9-9f86-0a786be205d2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:18.358543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:18.359444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 268/537] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:18.360284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.374920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:18.420153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.420941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.421338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.421713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.422538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.423790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.426157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.454926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 270/538] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:18.460845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.542632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96223c9-c623-4bfa-b47b-3281d66af237 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.595045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbca1b-3521-4d18-99d6-77c98faa37e3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.626156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.627144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.627551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.627970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.629251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.629733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.632309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.662939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:18.666536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.667435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.667889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 269/539] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:18.668209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.668669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.669732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=ca58b9ac9151436aad5a9546fdbe6c79) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.670264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.672019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59bbe7-4aa5-4db5-b5e6-71ad1cc33c97 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.674408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:18.681467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 271/540] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:18 2022] GET /identity/v3/role_assignments?user.id=ca58b9ac9151436aad5a9546fdbe6c79&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:18.795222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:18.838621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3566-ddaf-4ef5-bc26-0a511d89b0e3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:18.892821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:18.893773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:18.894255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:18.894743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:18.895770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=6d11c4cfca65434c9108393d11c1f7ee) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:18.896272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:18.897964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f75ae4-b11a-4f56-9964-43a96afb58b9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:18.904922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 270/541] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:18 2022] GET /identity/v3/role_assignments?user.id=6d11c4cfca65434c9108393d11c1f7ee&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:19.337933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeef64-941b-4598-b5de-514e923065bc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:19.338441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeef64-941b-4598-b5de-514e923065bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:19.338441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeef64-941b-4598-b5de-514e923065bc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:19.340010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 272/542] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:19.345252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:19.345671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:19.345850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:19.364483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:19.365680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:19.437715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 271/543] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:19.750502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdeef64-941b-4598-b5de-514e923065bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:19.880081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdeef64-941b-4598-b5de-514e923065bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:19.929697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:19.930603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:19.931043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:19.931340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:19.947763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:19.953019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:19.953911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:19.954738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 273/544] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:19.960222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:20.040009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:20.092891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a92bc-b3ac-4c58-82db-6357574df200 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:20.141446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:20.142058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.142334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.142625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.143290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:20.143587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:20.144948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:20.156163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 272/545] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:20.161802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:20.229012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:20.295095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e55f5-5c54-4c8c-bbf2-9f7ebd5983da None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:20.342889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:20.343522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.343785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.344058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.355251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:20.356706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0523868d-6edf-4e9e-9614-e0cdbbfb5fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:20.371895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:20.372834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:20.373691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:20.373691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:20.373691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:20.373691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 274/546] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:20.378830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:20.394459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65707-90ec-4a47-80aa-2e91115dc7b5 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.394786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65707-90ec-4a47-80aa-2e91115dc7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.395017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c65707-90ec-4a47-80aa-2e91115dc7b5 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.396472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 275/547] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:20.401570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.401847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.402119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.437836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:20.439192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:20.542769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a7ba-c2bd-4874-a70b-b23bb9386b59 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:20.571043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:20.571636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 276/548] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:20.588749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:20.589713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.590193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.590691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.591588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:20.592087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:20.593668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:20.607988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 273/549] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:20.614542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:20.670604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:20.751801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e939-7f6b-4bb3-a66e-ea77a766918b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:20.787043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:20.787680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.787954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.788220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.795811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:20.797260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:20.809338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:20.810880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:20.811896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:20.814764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 277/550] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:20.816325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:20.934927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035cdd3-2c6c-42c0-b5d8-13293a041b00 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:20.978173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:20.978811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:20.979091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:20.979362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:20.980097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:20.980399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:20.981839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401d2b-dc6e-4865-9fb2-e55314965e89 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.000726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 274/551] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:21.006380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.020814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c259-2c54-4f0c-bc27-e68ccd9e448c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.021104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c259-2c54-4f0c-bc27-e68ccd9e448c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.021378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c259-2c54-4f0c-bc27-e68ccd9e448c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.024471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 275/552] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:21.028565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.028851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.029126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.048121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:21.049314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:21.120518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.134087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.144373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe147a4-cf76-4aa2-8126-01e96782438a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.154624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 276/553] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:21.155543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.185221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.186271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.186925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.187450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.304914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:21.306601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de1d0-c2fc-43b7-a74b-06785e9dfa4e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.364196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5b22-04c1-45fd-9aab-c62859d11c6c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.402074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.402819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.403074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.403461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.415440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-877a6f00-dcbf-4b1f-8229-9b54d4c5b512 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:21.420796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:21.421750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:21.422632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 277/554] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:21.423801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea4eb-8fb5-4162-b495-e2e11bf45abf None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.423898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea4eb-8fb5-4162-b495-e2e11bf45abf None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.424128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea4eb-8fb5-4162-b495-e2e11bf45abf None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.429449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 278/555] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:21.429449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea4eb-8fb5-4162-b495-e2e11bf45abf None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.455531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.463231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 278/556] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:21.468919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.469272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.469739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.472479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.488026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:21.489217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:21.544566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea4eb-8fb5-4162-b495-e2e11bf45abf None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.563082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 279/557] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:21.564013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.586046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.587295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.587831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.588346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.589314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:21.589847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:21.591611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.606114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 279/558] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:21.607412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.673290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6502d4-c449-43b9-ba86-e3c2db8d51c0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.702905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.712477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.712852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.713135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.713410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.719966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.724368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:21.725898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.728464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6437c7-cca2-4077-86ba-ca353e27e62f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:21.741702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:21.743267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:21.744667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:21.744667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:21.744667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:21.744667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 280/559] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:21.745782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.768712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.769346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.769625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.769897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.781206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:21.786444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:21.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:21.788261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:21.792512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 280/560] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:21.793164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.852227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefba55-3f8c-43c1-866a-4dbabc793f25 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.889061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.889689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.889971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.890246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.893358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba045d5-2c9a-4bf9-a8b6-58623345449d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:21.901024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:21.902481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-4d9d5914-b297-496b-b8fb-1bc13e38e78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:21.917027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:21.918060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:21.918983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:21.918983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:21.918983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:21.918983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 281/561] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:21.919314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5013-511a-488b-a7b5-db5dc4bb304f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.919621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5013-511a-488b-a7b5-db5dc4bb304f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.919907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5013-511a-488b-a7b5-db5dc4bb304f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.921061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 282/562] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:21.921748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde5013-511a-488b-a7b5-db5dc4bb304f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.932341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:21.932937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:21.933216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:21.933481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:21.934126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:21.934418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:21.935825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:21.948101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 281/563] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:21.948933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:21.951606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:21.969150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.026364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde5013-511a-488b-a7b5-db5dc4bb304f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.048962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8468d8-d2f0-4eb8-895c-ce11c93bb045 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.059629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.060221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.060505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.060772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.061432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.061733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.063151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.071370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 283/564] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:22.072153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.090261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.090920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.091203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.091475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.101789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-852f6be9-d225-4543-8821-bf523c8b5553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:22.107400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.108347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:22.109200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 282/565] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.110366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.113508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.113508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.128842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:22.130034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:22.165459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.174505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7723e5-be1e-4057-8177-49e96e6e5c0a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.198565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 283/566] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:22.199404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.208327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.208934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.209212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.209481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.210111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.210412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.212215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.223799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.224116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 284/567] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:22.225002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.328330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f537c8-f382-4092-9b03-b72305d87cca None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.362387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb34717-f81c-4682-b2f6-6afe0c6ed7a9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.363568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.364201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.364488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.364767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.373267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.375132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:22.387379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.388285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:22.389163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 284/568] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.389650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.409196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.410196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.410860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.411140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.423022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.424474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:22.440948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.441912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:22.448176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:22.448176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:22.448176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:22.448176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 285/569] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.448176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.449890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.465023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.499632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17e07a-f403-45fa-a624-61982ddfd946 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.534036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.534694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.534980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.535263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.535933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.536237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.537614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.545792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 285/570] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:22.546615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.550887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12805e-f079-45c5-8914-23dfcb1c4341 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.584947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.585671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.585958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.586235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.594027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.595497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:22.607297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.608319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:22.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 286/571] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.611168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.649958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af78fa-dce3-43b8-9890-d6b023371820 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.673415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.676657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.685657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.686257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.686565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.686846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.687548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.687852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.689391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.702167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 286/572] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:22.703101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.712350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c03339-aa94-4ed4-bce1-f05dc69341d1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.747235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.747838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.748113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.748386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.749023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.749322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.750758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.762110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 287/573] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:22.762110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.804517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0b467-4792-496e-b3cf-ff222d7d3b26 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.838086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.838719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.838996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.839262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.854482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:22.859256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.860436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 287/574] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.861523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.874466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8475402-1770-4cec-8235-e4a3a648f60a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.897921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.910252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:22.910417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.910979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.911267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.911829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.922176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:22.923634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:22.938269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:22.939476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:22.940585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:22.940585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:22.940585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:22.940585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 288/575] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:22.940585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:22.964446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816e6f0-2d92-4a32-ac78-4bfc7eef5299 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:22.997557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:22.998165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:22.998448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:22.998753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:22.999487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:22.999796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.001303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.018489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 288/576] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.019408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.039783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245707-f75f-4b45-bd11-04086d8aedbb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.073571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.074165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.074440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.074735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.075456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=091613b4778d4946b0059334bdb70dd1) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.075761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.077184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.094556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 289/577] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:22 2022] GET /identity/v3/role_assignments?user.id=091613b4778d4946b0059334bdb70dd1&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.094556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.129785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b9298-6488-4ba6-b06b-159a381e08f2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.133429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.140411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.174042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.174826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.175182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.175527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.183991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.185570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:23.198464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:23.199740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:23.200939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:23.206732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 289/578] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:23.206732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.210059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d438f-1840-423e-8aed-0f85cfd8b63d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.249773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.250700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.251154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.251721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.252546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.253082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.254640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.263874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 290/579] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.264918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.313816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a2ed2-9093-4d6c-8f67-30d511ba499a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.349466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.350078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.350359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.350807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.351312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.351612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.353594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.370857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 290/580] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.371616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.397332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd66b0f-c486-4622-9a34-24b20c2671e6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.399218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.402049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.437080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.437713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.437993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.438266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.450188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.451681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:23.469678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:23.470737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:23.471718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:23.471718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.471718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:23.471718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 291/581] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:23.472569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.489763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9d1c-2389-457a-8c21-07e8b2f48f96 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.528339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.529032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.529313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.529631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.530704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.532404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.557009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 291/582] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.557009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.584786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797a29-d1d0-4c42-9e8f-155c881c7e27 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.618850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.619531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.619811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.620081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.635702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.637130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.644238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:23.652149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:23.653190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:23.654084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:23.654084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.654084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:23.654084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 292/583] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:23.654084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.666283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.668578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b419e3-fb3b-49df-af20-a4c1ca24ac11 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.703043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.703663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.703942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.704210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.712332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.713788 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:23.725935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:23.726977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:23.728150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:23.731691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 292/584] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:23.732396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.765533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dba33-9e41-49b9-a5e8-fdbf231992da None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.803111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.803654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.803933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.804201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.804863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.805789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.807184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.819290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 293/585] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.820111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.847214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d47e1-4c02-46a7-91e7-802034bfdf06 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.883109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.884174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.884751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.885225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.888638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.898262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.900535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.903468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:23.918016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:23.919031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:23.919908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:23.919908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:23.919908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:23.919908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 293/586] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:23.920552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:23.928669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a664fcb-e5fb-497e-9a2e-4bf65d38f119 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:23.966001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:23.966720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:23.967003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:23.967282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:23.968006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:23.968313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:23.969692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:23.981644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 294/587] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:23.982481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.037315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d659-6916-4a99-8231-b674e7470236 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.073622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.074148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.074425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.074725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.075455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.075757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.077193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.095866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bb347-4d3a-45e7-a025-8ee81702d8a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.096340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 294/588] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.097154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.128885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.139467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.140106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.140385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.140656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.147812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.149425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.157383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:24.163614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:24.165119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 295/589] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:24.166329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.211058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdbd5-1245-4bf9-a8ef-d317b1950738 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.246374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.247017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.247299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.247572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.248216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.248521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.249913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.262049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 295/590] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.262758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.274184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574648-22da-471e-8ec7-4ed4729e9608 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.310257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.310918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.311203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.311581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.319280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.320751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:24.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:24.333533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:24.334492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 296/591] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:24.335142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.374902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc85d6-ce0e-4fb5-9744-cb5c6e644311 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.387809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.389675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.412742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.413526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.413883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.414229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.425507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.427011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.445087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d2f0-d630-45f2-929f-bef78df4b2d4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:24.445255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:24.446907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:24.447996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:24.447996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:24.447996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:24.451376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 296/592] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:24.452263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.485635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.486240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.486531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.486821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.487517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.487823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.489272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.501700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 297/593] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.502407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.561562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088877e-98a8-4555-b682-17ddf124c07b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.596222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.596843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.597123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.597395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.604178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.604319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdf298-7f86-4fb4-b14c-3b6be952611b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.605620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.612681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:24.618309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:24.619572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:24.620478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 297/594] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:24.620980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.648857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.649564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.649848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.650125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.650885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.651192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.652596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.653525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.670408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 298/595] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.671471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.735704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9acae-096e-4194-a8ba-f96c10998700 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.777679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1bf35-29c7-4c70-998f-6532c7a48f56 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.778531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.779166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.779444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.779715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.780373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.780658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.782028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.793893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 298/596] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.794761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.821982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.822926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.823434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.823976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.840664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=7e45de636b0e4f35aa279718cc129923, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.842118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.858068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 299/597] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:24 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:24.866021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:24.887585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.892072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:24.913307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a23f0-400c-4e5a-b0f2-af9f8f9b8f06 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:24.963726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:24.964834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:24.964834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:24.965029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:24.965686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:24.966052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:24.967775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:24.982396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 299/598] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:24.988001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.006408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac868de-b6c0-4804-9670-4427f7c1d758 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.046663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.047664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.048188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.048562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.060232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.061831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:25.077856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:25.079174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:25.080280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:25.080280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:25.080280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:25.082326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 300/599] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:25.082326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.100336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa1e8-22d0-4a1a-81aa-5704d1415b83 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.142161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.142842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.143121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.143390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.151240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.152736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:25.165884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:25.166875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 300/600] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:25.168077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.175722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.181782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.201215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183aa69-8ce4-48de-95fc-99cdf6caf559 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.236995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.237813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.238200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.238649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/02a59e7ff79249eaa7f45cdbe15d0eaa/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.254779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=02a59e7ff79249eaa7f45cdbe15d0eaa, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.256486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.274656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 301/601] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:25 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/02a59e7ff79249eaa7f45cdbe15d0eaa/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:25.274656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.291629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa5ad6-ccee-449c-82ab-e233b9e6d365 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.344864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.345487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.345763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.346031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.346721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:25.347024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.348564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158250b4-d91e-4148-a2de-d98862c7ad2c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.363493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 301/602] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:25.364832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8d7e-aab9-4e3d-8e66-166ecedb4e5f None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.365115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8d7e-aab9-4e3d-8e66-166ecedb4e5f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.365444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec8d7e-aab9-4e3d-8e66-166ecedb4e5f None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.367038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 302/603] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:25.368917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec8d7e-aab9-4e3d-8e66-166ecedb4e5f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.417209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35312cf3-2e8c-4012-aa3b-968a8fb9afcd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.440985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.454412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.475215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.475945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.476329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.477798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.477956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:25.478261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.479737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015cba1-99ee-445f-96e0-67f266df9fef None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.492085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 302/604] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:25.493889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.494239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.494427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.499068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec8d7e-aab9-4e3d-8e66-166ecedb4e5f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.517415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:25.518932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:25.535947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.536827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.537272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.537686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.544764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.546398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:25.558366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:25.559495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:25.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:25.562628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 303/605] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:25.564008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.602438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 303/606] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:25.603610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.727791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e5a7d-2434-4746-83ee-35a0dc9342bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.745769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.764139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd0cf-14ac-48c9-a4af-013a053eb2b7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:25.795650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.796566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.797044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.797498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.798467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:25.798997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.800791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.807503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:25.817298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 304/607] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:25.823430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.829126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff00888-a323-4b06-95af-e99161361724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:25.835139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00888-a323-4b06-95af-e99161361724 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:25.835662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00888-a323-4b06-95af-e99161361724 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:25.837246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00888-a323-4b06-95af-e99161361724 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/ca58b9ac9151436aad5a9546fdbe6c79/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:25.869405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff00888-a323-4b06-95af-e99161361724 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=ca58b9ac9151436aad5a9546fdbe6c79, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:25.871223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff00888-a323-4b06-95af-e99161361724 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:25.897481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 304/608] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:25 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/ca58b9ac9151436aad5a9546fdbe6c79/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:25.931316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff00888-a323-4b06-95af-e99161361724 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:25.969448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39560f5d-9bde-4a4d-af80-e4ae5a114fd4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:26.007805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:26.008842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:26.009303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:26.009779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/6d11c4cfca65434c9108393d11c1f7ee/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:26.032902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=6d11c4cfca65434c9108393d11c1f7ee, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:26.034730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:26.059828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 305/609] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:25 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/6d11c4cfca65434c9108393d11c1f7ee/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:26.071702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff00888-a323-4b06-95af-e99161361724 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:26.130699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:26.131568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:26.131892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:26.132280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:26.150131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:26.155353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:26.156564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:26.157601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:26.158589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 305/610] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:26.166598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:26.190416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:26.302741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70f56-54bb-4a50-b03c-0c81d402acff None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:26.351307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb913c-0871-4007-adec-898724f76073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:26.352516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb913c-0871-4007-adec-898724f76073 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:26.352785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb913c-0871-4007-adec-898724f76073 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:26.353049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb913c-0871-4007-adec-898724f76073 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:26.353847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb913c-0871-4007-adec-898724f76073 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:26.354151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb913c-0871-4007-adec-898724f76073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:26.355604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb913c-0871-4007-adec-898724f76073 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:26.367788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 306/611] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:26.373744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:26.417045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:26.493049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfce736-3f0b-4899-9fcd-c5d7f28142bc None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:26.533536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:26.534173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:26.534456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:26.534756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:26.545649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:26.547140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:26.569565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:26.571548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:26.572447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:26.572447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:26.572447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:26.575066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 306/612] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:26.583610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb913c-0871-4007-adec-898724f76073 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:26.814148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:26.840672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb913c-0871-4007-adec-898724f76073 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:26.893395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:26.894016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:26.894299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:26.894594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:26.895245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:26.895549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:26.896915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:26.918048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 307/613] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:26.923106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:27.059438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:27.074551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8dd65-31bb-4f41-aa47-694c15870496 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:27.124346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:27.124978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.125254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.125524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.143599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:27.145062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-01bb7e10-772b-42f3-a2cc-5baf9a1dc7cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:27.159897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:27.161083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:27.162423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 307/614] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:27.167161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:27.304642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2eea-2a44-4a84-b988-2328d09bc6f9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:27.348659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:27.349279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.349601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.349875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.350691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:27.350994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:27.353180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:27.361947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:27.374104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9e8e-2c33-414e-9125-ca9568835e6c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.374323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9e8e-2c33-414e-9125-ca9568835e6c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.374627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9e8e-2c33-414e-9125-ca9568835e6c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.376274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 308/615] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:27.380068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 308/616] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:27.381710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.381985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.382258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.386055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:27.435548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:27.436761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:27.525261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bfde3-a37a-472b-9450-e674e84a5fe4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:27.569391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:27.570040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.570325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.570660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.573599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 309/617] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:27.592440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:27.593896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:27.613592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:27.615856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:27.618702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:27.618702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:27.618702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:27.618702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 309/618] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:27.621288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:27.702974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:27.735168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602e632-277c-45ab-a5d9-cfe2663998cc None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:27.775081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:27.775698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:27.775983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:27.776242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:27.776893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:27.777195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:27.778629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:27.790306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 310/619] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:27.795746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:27.889218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:27.965455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f36a3-2da5-4461-aa5c-6475a63508d7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:27.967580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.013093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.013720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.013996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.014266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.021327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.022967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-61cd064e-c77c-4318-83cb-0931b62114cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:28.035491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:28.036433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:28.037930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 310/620] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:28.038453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e377-4278-4dd0-9a54-927a3f1d4b17 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.038453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e377-4278-4dd0-9a54-927a3f1d4b17 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.038839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e377-4278-4dd0-9a54-927a3f1d4b17 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.040584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 311/621] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:28.045350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e377-4278-4dd0-9a54-927a3f1d4b17 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.093395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4e56d-2aa9-4ddc-b8bc-1a54760ad05d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.141130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.141798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.142072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.142338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.152365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.159404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-5b62c53d-fe65-42db-931f-ae77be092a12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:28.166584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:28.167558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:28.168951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 311/622] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:28.169671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.169671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.170016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.190747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e377-4278-4dd0-9a54-927a3f1d4b17 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.192466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:28.194843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:28.203352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.236404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.237637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.238192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.238772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.239927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:28.240440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.243152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.261820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 312/623] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:28.263375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.288550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 312/624] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:28.289875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.446869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9caa6-2ca0-410b-bfb5-194ae456aef6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.496516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.498764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.500100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.500605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.501585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:28.503417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.505968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4f932-a313-4254-992e-05200ae2bdb7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.508011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.528121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 313/625] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:28.536970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.575728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.584580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.592292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.592961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.593241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.593506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/091613b4778d4946b0059334bdb70dd1/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.616568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=091613b4778d4946b0059334bdb70dd1, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.618826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.635683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 313/626] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:28 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/091613b4778d4946b0059334bdb70dd1/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:28.636500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.685277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd0a2-cfab-4938-9129-0471794dd290 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.731912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.732626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.732891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.733227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.747821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.749275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-09460cab-1e80-4a5c-9b1d-c0bce53cc075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:28.766886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:28.768187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:28.769274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:28.769274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:28.769274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:28.769274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 314/627] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:28.769518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772ed7-03b5-469d-afa6-54309500e7df None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.769692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772ed7-03b5-469d-afa6-54309500e7df None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.769904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772ed7-03b5-469d-afa6-54309500e7df None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.775805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 315/628] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:28.775805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53772ed7-03b5-469d-afa6-54309500e7df None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.801107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e5379-afe2-4622-bfa4-e964d496b925 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.847068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.847701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.847972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.848238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.861664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e58de86c-e634-4e6c-8c38-88d2685a2368 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:28.866532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:28.868024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:28.869505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 314/629] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:28.870239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.870239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.870589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.879954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.888946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:28.890156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:28.903052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53772ed7-03b5-469d-afa6-54309500e7df None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:28.920666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:28.942013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:28.942656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.942932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.943207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.943857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:28.944190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:28.946116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d28c96-0e4d-4bca-b7c8-f7803ffb2bb1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:28.962690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 316/630] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:28.963611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 315/631] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:28.964119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683594f-888c-43a1-b240-21396dfc3dd0 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:28.964411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683594f-888c-43a1-b240-21396dfc3dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:28.964688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683594f-888c-43a1-b240-21396dfc3dd0 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:28.965463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:28.966188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 317/632] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:28.966942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683594f-888c-43a1-b240-21396dfc3dd0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.081698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc623d-d167-49ba-8f5e-1e87586ea20e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.114284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683594f-888c-43a1-b240-21396dfc3dd0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.127679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.128299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.128578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.128848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.129520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:29.129820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.131230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674a176-370d-4e58-9fe2-cd2576a6f754 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.140701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 316/633] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:29.142010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.142297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.142636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.161364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:29.162572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:29.166825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.167719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.168207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.168695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.180524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.182725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.190030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:29.201023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:29.202015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:29.203026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:29.204163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 318/634] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:29.204958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.207942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.249816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 317/635] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:29.254628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.321425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd4b2-6823-4e9e-b1a4-e721c2135d25 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.357397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.358026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.358357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.358706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.370483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.372047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:29.387194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:29.388215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:29.389743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:29.389743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:29.389743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:29.389743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 319/636] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:29.389743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.390980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dc743-f820-41f9-8b3d-93fd4e58fe94 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.437713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.438965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.439453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.440151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.441563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:29.442290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.444537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.451010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.471298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 318/637] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:29.472476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.499680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.505864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496fef-8642-4148-9cc0-da02214e02a5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.542002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.542660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.542943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.543216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.555691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:29.561276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:29.562318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:29.563214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:29.565884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 320/638] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:29.566893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.594734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8e3ad-d7cf-420a-a3ac-2cf893a3231e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.631578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.632478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.633255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.633609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.634460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:29.634791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.636162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.651855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 319/639] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:29.651855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.676446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7ca0-ebac-4b6e-ac20-dbb3c39f0ff5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.691628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.714924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.715529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.715866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.716138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.735418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.758740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d8e1-d8e1-4d82-9593-3b9e225cd5c3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.792613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.793518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.794146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.794279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.794440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.794711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.804929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:29.810119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:29.811143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 320/640] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:29.812010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.914761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 321/641] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:29.914761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:29.922914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e697d-ef6e-4ede-b0c0-cf57d835aaaa None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:29.928807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.959957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:29.960562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:29.960885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:29.961215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:29.962158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:29.962454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:29.963848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:29.969465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:29.972940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 321/642] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:29.973444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.029584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53328666-2a94-4332-a2bd-b553d6fc2bf9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.067212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.068031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.068408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.068791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.077244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.078701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.078904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ed55-0b3b-4f06-b4b3-11b60d3aed27 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:30.091775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:30.093010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:30.094189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 322/643] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:30.098166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.124301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.125041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.125418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.125779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.126559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:30.127079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.128536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.137531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 322/644] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:30.138412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.188880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.202992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.216829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4a86-8efc-4b7d-9730-1fe87989fa29 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.247354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ccb9a-e2ac-421a-b6e4-aff50c365c9e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.254141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.254921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.255299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.255671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.267280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.269128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.281289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.282026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.282402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.282800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:30.284312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:30.290749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:30.292101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:30.292101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.292101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:30.292101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 323/645] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:30.292101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.303199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.304865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:30.320062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:30.322181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:30.323380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:30.323380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.323380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:30.324075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 323/646] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:30.329020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.407346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6bd6f-bda9-4bad-aa38-931a92375238 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.433426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.444698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.445852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.446291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.446600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.447396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:30.447566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d575-e804-4471-94c2-c668da69f38e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.447821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.449457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.464203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.473756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 324/647] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:30.473756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.488779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.489413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.489696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.489968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.501864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.503360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:30.518451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:30.524819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:30.526011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:30.526011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:30.526011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:30.526011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 324/648] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:30.526011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.588683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82b3e5-7e72-49d0-a0fa-b53c48468762 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.623375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.623970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e5285-91ef-4194-a4e5-d385b85777d6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.624410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.624410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.624549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.625141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:30.625463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.626879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.639293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 325/649] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:30.644355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.658440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.659084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.659368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.659640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.660287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:30.660593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.661970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.674628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 325/650] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:30.675443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.699139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.704527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.763473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688a56-7ef9-4348-bbde-ca5f1ef73b2b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.784070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85bd3-e3e2-4baa-92ae-ad2987a8461a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.799014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.799478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.799758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.800024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.821859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.822463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.822762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.823043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.823663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:30.824366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.825320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.837710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 326/651] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:30.842677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.877845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.879324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:30.923121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.930230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:30.946385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605600-3e75-4d2d-bc85-6e64b6c1687c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:30.979627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:30.980192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 326/652] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:30.980306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:30.980436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:30.980707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:30.980859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:30.990970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:30.992458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:31.003636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:31.004539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:31.005441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 327/653] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:31.005935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.080082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2c6f4-3785-4dc1-b5d5-a16fc302b401 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.104143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ce54-ed3e-4560-939b-2253930dd187 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.114236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.115203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.115203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.115420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.123074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.124514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b63566d9-af52-4a38-a2e8-7c21b230c74e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:31.135710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:31.136675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:31.137633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 327/654] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:31.138429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac648e2-62e1-4c82-9f75-5e0defa18f9a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.138429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac648e2-62e1-4c82-9f75-5e0defa18f9a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.138429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac648e2-62e1-4c82-9f75-5e0defa18f9a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.139642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 328/655] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:31.140318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac648e2-62e1-4c82-9f75-5e0defa18f9a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.144640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.144947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.145463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.145738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.146011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.154544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.156015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:31.167409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:31.168545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 328/656] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:31.169807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.178640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.251321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac648e2-62e1-4c82-9f75-5e0defa18f9a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.267996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff41-5c99-4866-94f8-cc9d20b59b51 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.284053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.284678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.284955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.285226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.295827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.297255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.300608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.301207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.301488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.301761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.302457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:31.302794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.304225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5000c-d9a4-43ef-91a5-7a13f549a5dc None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:31.312148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:31.313123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:31.314177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:31.314177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:31.314177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:31.314177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 329/657] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:31.314833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.320830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 329/658] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:31.322116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.322401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.322705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.340596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:31.341782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:31.385201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.402677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.411666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 330/659] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:31.412475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.431117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9d3b4-8439-4218-91b7-263e2b60b636 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.467737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.468474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.468655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.468936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.469664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:31.469974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.471430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.489190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 330/660] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:31.489982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.524361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f3fe4-3bc8-41f8-8276-f119bc98ddac None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.560996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.561819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.562246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.562696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.563587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:31.564050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.565622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.583559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 331/661] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:31.584619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.599830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8134e1-c35b-4703-8fa3-2dedbb88c1ec None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.615946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.637595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.638200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.638483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.638786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.639430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:31.639735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.641100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.646918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.658123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 331/662] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:31.658123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.700575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c36f2-0502-40f6-bdc4-02b8ecf03b3a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.734897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.735556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.735838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.736109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.736838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:31.737147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.738602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.744685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 332/663] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:31.745548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:31.768399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dea9e-8e29-4966-bdc4-7b72620136f1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.808707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.809317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.809601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.809868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.859654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.863038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b816e9-2d5e-49a9-8e3d-b5ba8c70c8b8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:31.883752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:31.892830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.894451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:31.903115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:31.903896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:31.904009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:31.904279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:31.989930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:31.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.012554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 332/664] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:32.013973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.097822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.115770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 333/665] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:32.116706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.128636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec24f-ed7a-45c1-9fdd-a955909216b5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.139180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.165323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.165955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.166237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.166529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.174572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.176029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:32.188216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:32.189152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:32.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 333/666] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:32.190612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.228852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3f6b9-9f02-415c-9682-8051380179a9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.263354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.263985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.264265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.264535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.275314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:32.280106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:32.281027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:32.281876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 334/667] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:32.282612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.308993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340ae4-b995-4bb2-bc4a-0ae2fd8e74a8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.348396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.348939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.349217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.349491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.351984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.361831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.363287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:32.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:32.379865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:32.386535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:32.386535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:32.386535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:32.386535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 334/668] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:32.386535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.389006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.412722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe079145-406e-49ee-b476-dd5d5214e5c4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.448760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.449398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.449685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.449956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.460479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.462962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:32.479049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:32.480044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:32.480952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:32.480952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:32.480952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:32.480952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 335/669] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:32.480952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.494412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7cb23-9bd0-431b-8715-648160b775ad None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.530414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.531220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.531509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.531921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.532622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:32.532933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.534373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.548613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 335/670] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:32.549441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.596843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66839eb9-111c-4d2a-92b0-75b285fbc63e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.602491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.630574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.637347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.637957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.638312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.638541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.639188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:32.639490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.640859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.649990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 336/671] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:32.651340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.667043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d599c-9f47-4509-88fe-d39077a8b252 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.706958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.707574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.707912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.708187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.708832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:32.709133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.710496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.722486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 336/672] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:32.723603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.769664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13c050-c92f-4d14-8469-f3edffdc0ebf None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.804266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.805586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.805861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.806132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.806795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:32.807100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.808474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.824328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 337/673] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:32.824328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.836565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ff4f-ce1f-4a74-8c24-7ebf69edfcf7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.846064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.873057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.873585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.873867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.874137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.874893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=02a59e7ff79249eaa7f45cdbe15d0eaa) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:32.875206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.875936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:32.876631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:32.882952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 337/674] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/role_assignments?user.id=02a59e7ff79249eaa7f45cdbe15d0eaa&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:32.887486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:32.934293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3eb39-4735-440c-8673-fa292bbf2949 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:32.974461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:32.975145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:32.975422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:32.975698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:32.989373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:32.991682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:33.010734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:33.011791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:33.012717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:33.012717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:33.012717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:33.017056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 338/675] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:33.018047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.049450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eddfd1-09f2-496e-9155-c9bccfa2be45 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.099105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.099337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.099579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.099757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.114643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.117522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:33.132299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:33.134077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 338/676] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:33.135236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.154101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5695679-efc5-4263-aed2-0a29480e06bb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.158201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.178210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.202049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.203092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.203585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.203998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.212273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.214008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:33.226979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:33.228186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:33.229285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:33.230899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 339/677] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:33.231795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.258868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973809a8-e9c9-46d6-b3ea-e55982a6f599 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.294011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.294649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.295008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.295202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.295849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.296151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.297725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.310771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 339/678] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.311699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.343101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea74578-4794-45ca-8a21-c36b37edb209 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.381502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.382413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.383416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.383865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.384894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.385354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.387046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.400942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 340/679] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.402052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.411843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.422686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9343ba-6a3e-45d5-9d82-f806cf7cb1e7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.430868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.460168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.460843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.461173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.461705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.462404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.462750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.464186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.477204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 340/680] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.478261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.518923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308e927-9991-4aee-8388-df0f7260c020 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.558344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.559079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.559466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.559742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.568688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.570868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:33.583199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:33.584460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 341/681] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:33.585883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.610611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06896ce0-b850-497c-a445-9bec86931ab1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.647728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.648749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.649106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.649508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.650302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=ca58b9ac9151436aad5a9546fdbe6c79) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.650720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.652406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.659045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 341/682] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/role_assignments?user.id=ca58b9ac9151436aad5a9546fdbe6c79&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.660385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:33.683411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.699055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:33.714332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3066dc4-cc7a-4dc5-9f3a-461dfe513a21 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.756161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.757187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.758282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.758975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.759959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=6d11c4cfca65434c9108393d11c1f7ee) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.760504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.762364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.769740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 342/683] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/role_assignments?user.id=6d11c4cfca65434c9108393d11c1f7ee&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.888546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594862b6-90f5-4d11-8eff-a715dee74f41 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:33.927336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:33.927885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:33.928156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:33.928427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:33.929170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:33.929625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:33.931124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3b0c-c5fb-4187-a37f-f8b6f3f818b2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:33.953771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 342/684] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:33.959546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:34.093022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3caa1-6020-4a55-af28-3fbdea48d671 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:34.108887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b70f-bb7c-43f7-b3b2-548e8913da7f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.109390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b70f-bb7c-43f7-b3b2-548e8913da7f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.109491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b70f-bb7c-43f7-b3b2-548e8913da7f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.110835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 343/685] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:34.118304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.118628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.118910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.134449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:34.135150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.135433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.135703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.142105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:34.143573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:34.155092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:34.221507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 344/686] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:34.227928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:34.229473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:34.335018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 343/687] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:34.341647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:34.350845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:34.478004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e031d-88f6-491f-812c-8e514336132d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:34.522346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:34.523651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.523931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.524279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.540122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:34.541618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:34.557215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:34.558184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:34.559787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:34.559787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:34.559787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:34.559787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 345/688] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:34.564265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:34.622935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:34.668021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bea0cc-31fc-4810-877c-a77850206202 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:34.684021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:34.723957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:34.725343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.725626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.725899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.727109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:34.727413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:34.729536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:34.754228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 344/689] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:34.761509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:34.786841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:34.873019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc260ff7-e038-4900-b46c-d329648de077 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:34.936127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593be8e-d114-4d07-82c0-cda21bcd6aec None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:34.958876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:34.963234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:34.963891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.964178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.964456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.975287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:34.975843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:34.976642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:34.976933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:34.993911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:34.998800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.000291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:35.004374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:35.005341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:35.006800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 346/690] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:35.011246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:35.029425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:35.030381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:35.032463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 345/691] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:35.039985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.155560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd33f0-f385-4888-8c06-832d243d641f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:35.170598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:35.195303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:35.195911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.196189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.196464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.197147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:35.197451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.198845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.208224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 347/692] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:35.214598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.217785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedaff4-003c-410b-9bd8-915592e56bce None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:35.279747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:35.280277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.280553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.280820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.281513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:35.281811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.283306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.303501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 346/693] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:35.310160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.322314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b74184-ca99-43de-90a3-c0969127e61e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:35.366029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:35.367355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.367648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.367932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.380843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.386545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:35.406538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:35.407977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:35.408890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:35.408890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:35.408890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:35.408890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 348/694] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:35.410484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:35.416992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.461549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:35.548500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd412e-a0dd-47f8-bc6c-5a811af6b8bf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:35.584152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580b459-f062-4610-a539-2a62628a948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:35.584770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580b459-f062-4610-a539-2a62628a948b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.585053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580b459-f062-4610-a539-2a62628a948b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.585328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580b459-f062-4610-a539-2a62628a948b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.586005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580b459-f062-4610-a539-2a62628a948b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:35.586749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580b459-f062-4610-a539-2a62628a948b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.587955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580b459-f062-4610-a539-2a62628a948b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.600846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 349/695] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:35.602652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae5d3-8804-4bbc-9fb9-1c9685ecff6c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.602940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae5d3-8804-4bbc-9fb9-1c9685ecff6c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.603221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae5d3-8804-4bbc-9fb9-1c9685ecff6c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.605333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 350/696] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:35.613159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.614429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.614429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.646537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:35.646537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:35.662550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719567ef-5aa9-48a9-aeac-8dd2c06742df None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:35.734720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 351/697] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:35.735945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:35.737611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=091613b4778d4946b0059334bdb70dd1) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:35.742544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:35.762547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 347/698] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/role_assignments?user.id=091613b4778d4946b0059334bdb70dd1&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:35.774541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:35.990390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3814e7-f72f-480a-9b28-d1762bc0f8b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.027405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.028041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.028320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.028591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.035841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.037281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:36.048857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:36.051189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:36.053464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 352/699] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:36.057174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.130202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.182374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.195387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d8e7-895e-4783-834b-1c94db74e2c6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.242364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.243242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.243905 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.243905 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.244670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:36.245048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.247022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.248760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.270277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 348/700] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:36.279640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.296842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3e6ad-a744-4d13-80de-f26bbf56336c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.370303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.371219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.372881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.373315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.394744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.396534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e314334e-8225-49ea-9e71-5b094d422a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:36.413357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:36.415121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:36.416261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:36.416261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:36.416261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:36.417971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 353/701] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:36.424312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848637f-b4f7-439c-ac64-079823e3da02 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.424892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848637f-b4f7-439c-ac64-079823e3da02 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.425358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848637f-b4f7-439c-ac64-079823e3da02 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.427452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 354/702] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:36.428450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848637f-b4f7-439c-ac64-079823e3da02 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.461820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3164fae-7275-459d-95e9-fda7b205e881 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.497399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.498090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.498688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.498860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.520992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.549606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848637f-b4f7-439c-ac64-079823e3da02 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.582444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.584025 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.584205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.592496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.593402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.593933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.594434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.595358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:36.595888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.597718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f1081-ce60-4798-96a2-443560513fa9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.618850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 355/703] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:36.620816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.621390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.621884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.645436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:36.646673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:36.711523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 349/704] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:36.712440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.728668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 356/705] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:36.729542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.785235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.846336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54899ac9-1f3a-4851-9a56-5d9bf659e6e5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.848321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637ff4d-3ba1-4896-872e-863022649d69 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:36.870386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:36.886467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.887251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.887543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.887882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.893318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:36.894007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.894353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.894719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.895948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.897530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ea482-afee-49c6-8e9e-c44ee42be771 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.899017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:36.900476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dabc8566-53fa-405c-bd59-24c9bd732e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:36.917155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:36.918264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:36.919257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:36.919257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:36.919257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:36.922949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 357/706] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:36.924448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751dad-40bf-46d5-804d-3abaee98c01f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.924761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751dad-40bf-46d5-804d-3abaee98c01f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.925133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a751dad-40bf-46d5-804d-3abaee98c01f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.931258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 358/707] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:36.931258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a751dad-40bf-46d5-804d-3abaee98c01f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:36.933148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 350/708] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:40:36 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:36.934375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:36.942667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:36.943153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:36.983102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:36.984329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:37.093984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a751dad-40bf-46d5-804d-3abaee98c01f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.108322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 351/709] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:37.115725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.138067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.139002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.139118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.139391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.140134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:37.140493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.146727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.157525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 359/710] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:37.163114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.176306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.206919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.255108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99fce1-791d-4479-bc83-4f98cd94630c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.283114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df470b5e-0cf6-498d-808b-b27efdca7581 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.294196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.295173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.295912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.296422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.297664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.300543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:37.313156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:37.314537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:37.315573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.316667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 352/711] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:37.324570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.328539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.329445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.329906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.330370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.339605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.341429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:37.356959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:37.357924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:37.358880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:37.360520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 360/712] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:37.371757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.465285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.468463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028ddad-17e1-4406-b842-b892c7ea41c7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.505913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.506881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.507374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.507933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.509211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] RBAC: Adding query filter params (name=network) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:37.509918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.511747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.514481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.517392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 353/713] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:37.518338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.521131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794a070-c5f8-45a6-a229-41d031751b1a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.558783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.559750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.560317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.560821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.561768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:37.562382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.564143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.577695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 361/714] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:37.579015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.628687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb379-56dc-4e98-a38e-e924a0113f27 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.664260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.664897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.665177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.665451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.666045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.667462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:37.676530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:37.677566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:37.678452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 354/715] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:37.679418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.699312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34368aa-d471-4bfd-b341-905ad1abb85b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.718328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.751581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.752639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.753018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.753392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.754386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] RBAC: Adding query filter params (type=network) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:37.754804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.756959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.764779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 362/716] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:37.766345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:37.803403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:37.838010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745ba08-c432-4197-85ac-daddbe4dee60 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:37.919192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:37.920059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:37.920523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:37.920960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:37.921898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:37.922411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:37.924134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:37.935643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 355/717] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:37.966164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32ea1-f905-49c7-8d84-8949f9a743af None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:38.009904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:38.010889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:38.011314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:38.011731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:38.012551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:38.013000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:38.014675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758ac7-ce29-4448-b3d1-1753462a1053 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:38.020010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 363/718] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:38.025908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:38.143850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:38.156170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea561c8-9025-4631-b9c3-12c641ca10f2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:38.193309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:38.194149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:38.194666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:38.195081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:38.195953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:38.196411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:38.198101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb976dc9-b783-4e34-aa10-77a46e6f461f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:38.208491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 356/719] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:38.409375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ed-7953-49c0-a2cd-edd886ab2060 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:38.409889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ed-7953-49c0-a2cd-edd886ab2060 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:38.409967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b550ed-7953-49c0-a2cd-edd886ab2060 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:38.411608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 364/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:38.418056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:38.418400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:38.418632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:38.443183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:38.444469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:38.520640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 357/721] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:38.834840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b550ed-7953-49c0-a2cd-edd886ab2060 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:38.951724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:38.973274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b550ed-7953-49c0-a2cd-edd886ab2060 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:39.018197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:39.018923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:39.019208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:39.019483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:39.036071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:39.041186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:39.042150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:39.043030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 365/722] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:39.048514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:39.159170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:39.182549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306ce0f-3abe-45e9-aeee-76adb71e2ac3 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:39.227241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:39.227849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:39.228129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:39.228400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:39.229070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:39.229374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:39.230870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:39.239472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 358/723] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:39.245897 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:39.377338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b926cf6-0d7b-4064-a4ea-4875cf43ed32 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:39.424772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:39.425445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:39.425705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:39.425942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:39.435564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:39.442833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:39.444319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:39.465835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:39.466853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:39.467842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:39.467842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:39.467842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:39.467842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 366/724] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:39.475621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:39.615485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf56ad-f8f4-406f-91dc-8ea0c877d1be None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:39.659878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:39.660415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:39.660697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:39.660971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:39.661616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:39.661922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:39.664049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:39.680319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 359/725] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:39.685894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:39.799437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9db43f-893d-4c82-af15-6ff34eeaf6c7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:39.846593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:39.847232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:39.847507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:39.847780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:39.857019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:39.858478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-41f2201d-da99-4bff-a502-80a7ab7abd85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:39.874012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:39.875040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:39.876012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:39.878048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 367/726] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:39.883595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:39.902828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.066446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e73dc-4215-4e6d-ab5f-45f381075c13 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.067142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e73dc-4215-4e6d-ab5f-45f381075c13 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.067686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e73dc-4215-4e6d-ab5f-45f381075c13 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.070106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 368/727] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:40.072535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae69cd-4913-41b0-b4cb-b29d7bb5558f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:40.076342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.076865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.077340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.103710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:40.105228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:40.123481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:40.124107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.124389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.124664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.125411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:40.125718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:40.127205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:40.162223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 360/728] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:40.170973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:40.173580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.204103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 369/729] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:40.277070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.323070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb19-25a3-44e9-930d-8105dbf6c135 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:40.370823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:40.371440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.371719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.371992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.453881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:40.455373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20287051-d1bb-4466-8cba-29b7bc56e5eb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:40.544944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.583622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 361/730] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 11:40:40.589876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:40.600205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bd5b-ae3e-49be-9e23-3c854d8666b9 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.600491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bd5b-ae3e-49be-9e23-3c854d8666b9 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.600880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bd5b-ae3e-49be-9e23-3c854d8666b9 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.602565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 362/731] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:40.608483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.608761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.609040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.637157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:40.638611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:40.726970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a86d8-a00e-454d-b4c3-97f247e9a506 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:40.742420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 363/732] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 11:40:40.743767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:40.776097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:40.776851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.777055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.777333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.789198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:40.790874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2bce9593-65b4-4a10-ab33-965e1357a63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:40.807020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:40.808045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:40.808962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:40.808962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:40.808962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:40.812887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 370/733] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:40.814253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4d158-c2ca-4265-9b52-54a8601e7f59 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.814560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4d158-c2ca-4265-9b52-54a8601e7f59 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.814841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4d158-c2ca-4265-9b52-54a8601e7f59 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.820903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 371/734] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:40.820903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4d158-c2ca-4265-9b52-54a8601e7f59 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:40.861374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.863079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:40.871386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1cbad-f1de-4ea1-a410-11ca130fa10f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:40.908869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:40.909538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.909821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.910094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.921675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e9308d42-0306-4db6-9b35-46ced57fd807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:40.926994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:40.927952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:40.928892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:40.933056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 364/735] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:40.934449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.934761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.935086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.945173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4d158-c2ca-4265-9b52-54a8601e7f59 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:40.955421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:40.956575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:40.984262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:40.985349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:40.985840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:40.986367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:40.987320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:40.987831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:40.989647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.003810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 372/736] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.004948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.037717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 365/737] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:41.037717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.131422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.140614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2a3d-1528-4685-8f21-f731354d663e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.149185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.173562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba578d-839d-460b-b94a-40a356bd97b7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.188697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.189412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.189700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.189979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.190689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:41.190998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.192627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.202892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 373/738] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.208504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.217016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.217657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.217936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.218740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.228051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.230173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:41.247430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:41.248372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:41.249280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 366/739] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:41.249785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.324807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab658cf5-4e5c-47a0-bc53-a690876a4be1 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.358025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063699db-6b95-4de7-a57a-c6e69b402479 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.361727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.362389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.362713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.362988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.373865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:41.379916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:41.380823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:41.381666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 374/740] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:41.383066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.397691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.398547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.398896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.399197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.400389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.410245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.411765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.418610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e61ca5c9-7d99-4eaf-a224-f35cfa1e44d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:41.427362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:41.428494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:41.430263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:41.430263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:41.430263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:41.431652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 367/741] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:41.432825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4a48-7d61-41f9-8558-4d78a43ccb9e None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.433112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4a48-7d61-41f9-8558-4d78a43ccb9e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.433391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4a48-7d61-41f9-8558-4d78a43ccb9e None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.434636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 368/742] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:41.435316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4a48-7d61-41f9-8558-4d78a43ccb9e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.486124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acae034-d84f-403f-a3f7-eee2d3860263 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.519565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.520161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.520442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.520712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.521390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:41.521690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.523393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.536495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 375/743] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.537521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4a48-7d61-41f9-8558-4d78a43ccb9e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.537825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.574402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.575037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.575331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.575601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.576236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.577601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.594344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 369/744] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:40:41 2022] POST /identity/v3/services => generated 240 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:41.595180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.629608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.661287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45d44f-b43c-4810-a95a-d7f1c556454b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.662041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.720490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.721850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.722375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.722783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.723448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49a8f5-10a4-453c-bffd-f35131b29315 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.723574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:41.723779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.725584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c31079-3df2-42b3-87b1-4d47e84eb780 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.736674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 376/745] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.738148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.738555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.738910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.759887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:41.761099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:41.766673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.768061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.768061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.768061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.768456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:41.768755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.770557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.784677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 370/746] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.785777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.841665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 377/747] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:41 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 11:40:41.843500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.895512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206fb7-9534-4175-9a2e-118d9507f52e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:41.912996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.934044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:41.936037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:41.936814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:41.937150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:41.937556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:41.938359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=091613b4778d4946b0059334bdb70dd1) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:41.938768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:41.940688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:41.948164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 371/748] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/role_assignments?user.id=091613b4778d4946b0059334bdb70dd1&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:41.949241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:41.987649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccac7f1-3c7c-47a0-8893-f9dbc4e818de None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.033941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.034969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.035426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.035872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.048018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.049870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:42.067480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:42.068515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:42.070133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:42.070133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:42.070133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:42.070308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04c21-4cb6-4089-b0a0-7fef6544cd5a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.072419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 378/749] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:42.078128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.108451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.109224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.109569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.109911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.119155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.120700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:42.133487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:42.134500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:42.135466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:42.138684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 372/750] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:40:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:42.140635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.223851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.234923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a2dad-9cc6-43b1-9154-ad32a8c55de4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.271976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.291643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.292242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.292520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.292790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.293431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:42.293871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.295262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.307692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdef4-39cc-4b44-85b2-7d6a2646cda0 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.307984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 379/751] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:42.313226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.349771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.350385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.350811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.351076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.351929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:42.352234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.353657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.372358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 373/752] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:42.373183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.438952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e822-b89e-423b-b67e-346f29efd5ac None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.478631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.478889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.479181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.479405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.480088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.481446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.500468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 380/753] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:40:42 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:42.501499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.523604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.533890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968967-74d4-4bf7-ba47-d6298097d48a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.548381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.586763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.587834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.588315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.588783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.597531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.599603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:42.613976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:42.615146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:42.616331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:42.618602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 374/754] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:42.624612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.683389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d89070-25fb-4b37-b66b-fe1f7c6a197a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.728625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.729364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.729523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.729864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.740944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8e4b9-e15e-48fd-80a9-e11792677d54 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.778988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.779945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.780373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.780889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:42.781768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:42.782244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.783974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.804457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:42.805999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:42.817366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 375/755] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:42.819306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.837725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.845521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:42.920222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 381/756] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:40:42.925552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:42.943190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc59e-12bf-4c65-bf34-e7e78c9137c9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:42.985455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:42.986369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:42.986879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:42.987297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.008047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.009896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:43.035928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:43.037139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:43.038778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:43.038778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:43.038778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:43.046950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 376/757] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:43.047599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22650-40ff-4645-8880-a7d137f94424 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.052593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.088557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.089198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.089475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.089744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.094538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.105527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.107712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:43.128904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:43.130114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:43.131267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:43.131267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:43.131267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:43.131267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 382/758] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:43.136023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.183323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.231670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0946d7-1f5b-4831-b5a8-05a20f389909 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.271794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.272346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.272842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.273138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.273774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:43.274074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.275875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.289482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 377/759] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:43.295070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.399270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9d491-c001-44e0-b7bd-ec0143cad2a3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.409425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752879fb-b5ba-4c92-964f-aeb42dde2519 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.433745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.446691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.447905 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.448252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.448949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.458222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.458222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:43.478756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:43.479736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:43.480646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 378/760] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:43.488544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.490797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.491455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.491728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.491998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.492643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:43.492943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.494313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.519774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 383/761] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:43.525996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.629602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.676170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2847a3-e30c-44bc-8105-4f4b8be341b6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.715680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.716413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.716760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.717103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.717909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:43.718359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.719868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.733909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 379/762] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:43.742351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556a2d3-9357-4620-8f60-4096e497ce0b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.744195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.784348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.784915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.785195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.785470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.797509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.798994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:43.819853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:43.821114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:43.822006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 384/763] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:43.832049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:43.843831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.926332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a359ef-460a-4e66-a390-4a1417f21f52 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.963590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:43.964259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.964542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.964717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03917ba-5689-46c5-8416-43172356d6a6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:43.964865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.967060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=7e45de636b0e4f35aa279718cc129923, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:43.968508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad7e3a-3915-48c1-8673-4c3cd3ebe4b2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:43.974237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:43.985735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 380/764] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:43 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:43.987392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28185-1a53-4918-8521-a4830eb11da0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.987756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28185-1a53-4918-8521-a4830eb11da0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.988122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28185-1a53-4918-8521-a4830eb11da0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:43.989875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 381/765] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:43.995718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:43.996019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:43.996288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.014063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:44.014726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.015008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.015280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.016005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:44.016318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:44.017809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:44.031287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:44.032556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:44.045655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 385/766] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:44.052135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:44.181805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 382/767] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:44.219258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23c7e6-4204-4853-a25c-cf44e4d9fa25 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:44.260295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:44.260923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.261265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.261545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.262336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=a17fa3464f414b2da364930144e85fc4) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:44.262683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:44.264136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:44.270415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 386/768] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:44 2022] GET /identity/v3/role_assignments?user.id=a17fa3464f414b2da364930144e85fc4&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:44.467342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:44.508382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:44.591332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fcf98-34f6-475a-a4cb-845ec3862141 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:44.626342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:44.627011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.627290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.627560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.628142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:44.629514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-c324ad59-22e7-4efc-8ee1-69cc292691e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:44.638998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:44.639999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:44.640840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 383/769] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:44 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:44.645939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:44.676694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd56ba1-9f83-4b0c-839c-4050b210ad4b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.677152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd56ba1-9f83-4b0c-839c-4050b210ad4b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.677629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd56ba1-9f83-4b0c-839c-4050b210ad4b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.680128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 384/770] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:44.686132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.686747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.686914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.707686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:44.709034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:44.714878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:44.817396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd307f-c9b6-4fec-8d80-93983b7979f0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:44.853151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 385/771] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:44.860237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:44.860865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:44.861140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:44.861407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:44.862039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] RBAC: Adding query filter params (name=network) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:44.862339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:44.863732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:44.873728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 387/772] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:44.875013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:44.915103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:45.013938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bcde-95a1-4f38-99c4-d354b200494c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.062861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.063478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.063758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.064029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.064720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] RBAC: Adding query filter params (type=network) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:45.065024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:45.066398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:45.074670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 386/773] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:45.080370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:45.182928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:45.189031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39896c1-2e56-45ae-bf9e-28057e10a873 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.223827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.224642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.225278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.226178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.227197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] RBAC: Adding query filter params (interface=public, service_id=8345251d34d94a659949ea18bd9ca7f2, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:45.227515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:45.228879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:45.237052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 388/774] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/endpoints?service_id=8345251d34d94a659949ea18bd9ca7f2&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:45.243300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:45.275926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:45.424664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:45.439707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc96be7-d0a9-43b5-97b9-e69d9c567b96 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.447541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c9d7-a2fc-4772-93f8-1375da584eca None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.494919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.495558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.495839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.496117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.507917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:45.509023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.510698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.511379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.512038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.513304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:45.514953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-75221b82-c629-4ca2-b71f-9ea31ff620dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:45.515218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:45.516139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:45.516984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 389/775] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:45.518024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f9cd-33f2-461a-b1ff-770220dd3833 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.518130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f9cd-33f2-461a-b1ff-770220dd3833 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.524939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f9cd-33f2-461a-b1ff-770220dd3833 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.524939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 390/776] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:45.525782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.526069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.526351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.533858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 387/777] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:45.537453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:45.558531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:45.559818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:45.592756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:45.651204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343acc0-c039-46a4-8872-ad7464c2fb0e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.689493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.690466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.690913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.691512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.692362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:45.692837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:45.695244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:45.705205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 388/778] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:45.710611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 391/779] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:45.711977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:45.731985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:45.860883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa44c-e5f7-4b62-a10f-583eb3828580 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:45.909917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:45.910573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:45.910855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:45.911125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:45.931765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:45.933257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:45.951226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:45.952198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:45.953086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:45.953086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:45.953086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:45.953086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 389/780] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:45.959138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.002825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.082873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.094289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bff6b-a0b0-4f06-9ece-63ef3edd09df None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.129988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4cb3-80d5-4c3b-8939-a016fc849439 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.170863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.171504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.171879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.172157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.172750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.174135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.179035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.179659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.179934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.180204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.180853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:46.181152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.182543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:46.183786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:46.184741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:46.185610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 390/781] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:46.195804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.213643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 392/782] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:46.218957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.240554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.359922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeeb574-d366-48cb-93d0-c64d7360d48a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.392677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.400036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.400652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.400935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.401208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.401961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:46.402269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.403719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.408892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 391/783] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:46.414717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.440679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836205aa-3481-44b7-b1c2-42aef67b7fc4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.501109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.501769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.502058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.502334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.511074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.512533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.527315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bf04b-5dfd-4b34-8bda-f961479cd56b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:46.527928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:46.528832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:46.529792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:46.530783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 393/784] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:46.540331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.606990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.607605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.607883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.608156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.610087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:46.610396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.611812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.618082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 392/785] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:46.623590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.654884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.686260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c50db-e623-4a3a-b931-f999fef97d76 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.711622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.722643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.723389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.723669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.723944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.724680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:46.724982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.727556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed98b4-bae4-462d-85de-9f128d39bcd4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 394/786] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acd5-e6b1-4eaf-b049-1f253d882846 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acd5-e6b1-4eaf-b049-1f253d882846 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acd5-e6b1-4eaf-b049-1f253d882846 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 395/787] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:46.754624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7acd5-e6b1-4eaf-b049-1f253d882846 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.767991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e243-051c-4331-a17c-d98fe4450feb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.805165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.806140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.806593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.807009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.807888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] RBAC: Adding query filter params (interface=public, service_id=a7ff1141b0024457b5a3c5adbc1ba0f8, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:46.808341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.809914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c13df-3359-4105-8a39-73a49036177f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.818325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 393/788] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/endpoints?service_id=a7ff1141b0024457b5a3c5adbc1ba0f8&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:46.820383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.820946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.821427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.840720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:46.842233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:46.882068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7acd5-e6b1-4eaf-b049-1f253d882846 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:46.921505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:46.925141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 394/789] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:46.926125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:46.933687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:46.934335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:46.934647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:46.935779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:46.962656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:46.964160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ba3f049d-fb46-4dde-ac8b-c407e82699b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:46.990997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:46.992091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:46.992988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:46.992988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:46.992988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:47.000222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 396/790] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:47.003997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e73b-0741-4327-855c-fb2a8244d3dc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.004517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e73b-0741-4327-855c-fb2a8244d3dc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.004958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e73b-0741-4327-855c-fb2a8244d3dc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.007034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 397/791] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:47.010289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e73b-0741-4327-855c-fb2a8244d3dc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.022895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.068567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118ccfb-3734-40e0-9d95-68540d65790c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.107133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.107749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.108032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.108302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.108910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.110857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6335b-1870-49fc-b945-90c422947279 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.126886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 395/792] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:40:46 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:47.128164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.128452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.128730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.153411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e73b-0741-4327-855c-fb2a8244d3dc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.171734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:47.172982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:47.213637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.215303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.216067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.216859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.217805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:47.218411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.220298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.244960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 398/793] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:47.250553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.297098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 396/794] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:47.297899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.310038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.390964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.394257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074afd3d-ee99-43e0-80ef-877108254874 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.435497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d35e3-f81a-4e3f-8cf0-e355069e005a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.437210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.439056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.439553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.440055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.456532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:47.463368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:47.464323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:47.465167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:47.468170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 399/795] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:47.475005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.479720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.480473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.480724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.481058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.490396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.492280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:47.516738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:47.517727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:47.518681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:47.521144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 397/796] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:47.530095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.658797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973c29-efe9-48ce-92da-605c7c02d4f8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.670687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.697325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.703541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cc08b-c451-4b67-8e7b-06f44b0a9656 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.707827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.708627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.709056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.709501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.711112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:47.711577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.713656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.729998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 398/797] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:47.731300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.744381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.745137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.745852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.746236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.747039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:47.747748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.749491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.762591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 400/798] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:47.763619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.861105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027940a-b4a0-4058-a0d0-6d227fdc0663 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.877100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beb9c6-501e-41b1-a4d5-709c5c865dfc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:47.909154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.910409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.910872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.911530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.917252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:47.918113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:47.918599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:47.919047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/091613b4778d4946b0059334bdb70dd1/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:47.921193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=091613b4778d4946b0059334bdb70dd1, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:47.922812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:47.928461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:47.935147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:47.936492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:47.937451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:47.941726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 399/799] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:47.942734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 401/800] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:47 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/091613b4778d4946b0059334bdb70dd1/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:47.942861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.944356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:47.981227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:47.985401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.068074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61eee9-0938-475e-a628-0cad8247eeae None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.093913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ea813-a550-46ee-94c1-adc7de9b6d0a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.121523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.121523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.121523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.121523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.141379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.143011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:48.174596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:48.175574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:48.176519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:48.176519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:48.176519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:48.185741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.186392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.186712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.186982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.187782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:48.188094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.188508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 400/801] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:48.189496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.191165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.220842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 402/802] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:48.230227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.294337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.327435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd25-7acf-4e49-abba-b756f71149e6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.354233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.367100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.368040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.368622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.369094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.370018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:48.370632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.372464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592702e-35a4-4d3d-a45e-26414fabb799 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.377961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5df4d-962e-4e67-82f0-1475ab96c2fd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.386432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 401/803] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:48.388139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f105-c1ad-477d-89ab-abb16ab5b758 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.388632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f105-c1ad-477d-89ab-abb16ab5b758 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.389122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0f105-c1ad-477d-89ab-abb16ab5b758 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.390942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 402/804] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:48.391904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f105-c1ad-477d-89ab-abb16ab5b758 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.417093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.417736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.418022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.418783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.432791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.434869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b2d12068-ae21-4c84-be48-56198146465d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:48.456644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:48.457619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:48.458540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:48.458540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:48.458540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:48.465821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 403/805] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:48.465821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.465821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.465821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.488844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:48.491110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:48.515224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0f105-c1ad-477d-89ab-abb16ab5b758 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.552173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.552803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.553757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.554684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.562423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.564037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.567740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.579555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 404/806] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:48.579724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:48.580811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:40:48.581764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:48.583531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.587089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 403/807] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:48.587935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.704390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97019926-8ab5-409d-adfd-fa16980d4ad7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.743800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.746590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.747197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.747479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.747752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:48.748490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:48.748796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:48.750239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:48.770202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 404/808] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:48.776499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:48.828694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab1a52-9910-482d-9b77-e598b4153bed None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.839401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:48.892635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e9aaa-2275-414c-a1f0-c92b90427a94 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:48.930434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:48.931154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:48.931441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:48.931710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.000229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.001623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.002160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.002669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.003539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:49.004045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.005731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.009383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.010863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.019784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 405/809] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:49.020977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.072978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.128511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 405/810] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:40:49.129798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.136312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720f188-c369-492e-b280-bce6f452d82f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.164445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.174357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.175108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.175489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.175861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] PATH_INFO: `/v3/services/network` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.176560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.177983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:49.189335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:49.190539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:49.191396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 406/811] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:49.192344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.245166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ca126-d335-4908-9ca2-0a02045055e6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.283451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.284728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.285164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.285590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.296754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.299076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.301399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3249ef-8307-4da6-b287-2e7c42ee3735 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.340203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c522-544e-4034-bb50-186766807808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.341034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c522-544e-4034-bb50-186766807808 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.341417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c522-544e-4034-bb50-186766807808 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.341787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7c522-544e-4034-bb50-186766807808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.359111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7c522-544e-4034-bb50-186766807808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.360841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7c522-544e-4034-bb50-186766807808 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-acb7c522-544e-4034-bb50-186766807808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:49.379014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:49.380182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:49.381080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:49.381080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:49.381080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:49.383790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 407/812] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:49.384853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c522-544e-4034-bb50-186766807808 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.435151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:49.448514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:49.449537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:49.450497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 406/813] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:49.451130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.472924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.523190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7c522-544e-4034-bb50-186766807808 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.563097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5ba28-d2b9-41f4-a6bb-c9e326c3f8c1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.572041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.572933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.573797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.574259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.575565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] RBAC: Adding query filter params (name=network) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:49.576312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.578149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.591191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 408/814] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:49.591191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.615594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.617069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.617521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.618007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.619685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:49.620416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.622628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.639491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 407/815] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:49.640782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.725458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232646-ba23-47db-aa87-0a34d605af09 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.730321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.736576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.759451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3fe5d-989b-4fec-b7f6-5cc3bf10c3fe None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.767435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.768261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.768636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.769004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.769847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:49.770228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.771912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.794021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 409/816] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:49.795215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.803705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.804824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.805102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.805371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.806287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] RBAC: Adding query filter params (type=network) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:49.806415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.807972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659471b-e58c-4056-8fd7-5fee1f885c50 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 408/817] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48119-73f6-49f0-903d-b9d8fe6f4565 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48119-73f6-49f0-903d-b9d8fe6f4565 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48119-73f6-49f0-903d-b9d8fe6f4565 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 409/818] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:49.817924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d48119-73f6-49f0-903d-b9d8fe6f4565 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.903393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd490493-b42c-4bdf-9600-07b0c3d82ec4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.925358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d48119-73f6-49f0-903d-b9d8fe6f4565 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:49.938057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.938774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.939050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.939322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.946740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.948157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.959241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:49.960342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:49.961778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:49.962803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:49.963254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.963254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.963254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.966686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 410/819] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:49.967382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:49.974220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:49.976004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:49.977974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-44977958-6b72-4359-a9eb-35291c6663be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:49.992059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:49.993069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:49.993971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:49.993971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:49.993971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:49.993971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 410/820] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:49.994711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:49.994711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:49.994885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:49.998813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.014140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:50.015345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:50.078344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56724d5-ce31-4f27-be6e-ebadf3d97062 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.081068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 411/821] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:50.081910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.125795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.126481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.126858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.127197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.127932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.129443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.155192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 411/822] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:40:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:50.156236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.214097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704306d1-d93b-4e30-ac2d-3d76430a860d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.248644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.250247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.273832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.274458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.274763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.275030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.276311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:50.276618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.278243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.294176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 412/823] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:50.301291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab00b7d-2c32-4229-acca-4ac1e41e269c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.307133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.375895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.376900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.377482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.378784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.379660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:50.383040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.384717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.410056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 412/824] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:50.412068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.481903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e164c5c-fbb2-43c5-a219-34493818ba5e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.519847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.520449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.520729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.520995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.521723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:50.522714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.524334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e237-9029-42ee-84cd-043e9f33893d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.529335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cba09-e244-48a2-a3af-f6f103370546 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.530138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 413/825] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:50.531317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3432cf-d14d-4f8d-a470-b2372d1830a0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.531664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3432cf-d14d-4f8d-a470-b2372d1830a0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.531940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3432cf-d14d-4f8d-a470-b2372d1830a0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.533112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 414/826] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:50.533807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3432cf-d14d-4f8d-a470-b2372d1830a0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.571394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.572046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.572326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.572599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.580280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.581738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.586220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.589949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a06ad9d7-72f9-427f-b996-6ca6da66cbee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:50.599178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:50.600361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:50.601290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 413/827] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:50.601780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.601906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.602216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.621141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:50.622405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:50.655027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3432cf-d14d-4f8d-a470-b2372d1830a0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.695055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.695272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 414/828] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:50.695788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.703321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.704002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.705061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.705873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.708371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:50.719408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:50.720469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 11:40:50.721341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:50.722084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 415/829] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:50.730634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.831534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518278-4f07-40ce-a75d-ef7d2ced9ec6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.861822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.869351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.870047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.870280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc17188-ec67-4880-82e2-c3851ffc6d32 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:50.870488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.870810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.871580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:50.871950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.873466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.881033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:50.892098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 415/830] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:50.894070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:50.917135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:50.917852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:50.918322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:50.918729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:50.919612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:50.919955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:50.921414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:50.926898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 416/831] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:50.932519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.013120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a1d-d8b1-4b54-95e7-f39e5b5cae0c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.047872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.048540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.048823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.049046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d587-da06-43f4-a67f-c96566e59fc6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.049159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/a17fa3464f414b2da364930144e85fc4/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.066383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=a17fa3464f414b2da364930144e85fc4, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.067845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.082378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 416/832] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:40:50 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/a17fa3464f414b2da364930144e85fc4/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:40:51.083614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.087840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.088717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.089142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.089573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.090374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:51.090706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.092131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.098395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 417/833] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Sep 12 11:40:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:51.104014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.138445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:51.160852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:51.218367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0cef-cb8c-4975-be3e-8be0e7b7cceb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.254435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76513f-f6f8-4abc-9bf8-9c9b9422c137 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.261806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.262686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.263074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.263451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.278040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:51.283645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:51.284603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:51.286416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 417/834] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:51.292542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.302223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.302887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.303348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.303600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.304265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.305649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.332587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 418/835] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:40:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:40:51.433144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:51.439315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8f2a-4a86-4e85-8366-ec121540ce1e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.488269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.489126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.489548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.489977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.491008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:51.491459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.493080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.503082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 418/836] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:51.513140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.608822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:51.617984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53ed79-812c-42f4-a99a-789e7ac055c4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.652848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.653479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.653760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.654033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.665048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.666505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:51.681635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:51.682626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:51.683526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:51.683526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:51.683526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:51.683526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 419/837] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:51.688746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.791417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bcda4-edd8-454f-b3a7-7db163bb7175 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.829937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.830579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.830865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.831143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:51.831796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:51.832101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:51.833523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:51.846805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 419/838] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:51.852368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:51.956915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457346-c52e-47de-8547-5661d85108c3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:51.992218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:51.992852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:51.993137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:51.993414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.001058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:52.002526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:52.014171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:52.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:52.016081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 420/839] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:40:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:52.021341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:52.041810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:52.128967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b233b-78ec-44ac-929b-8bdd30d497a8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:52.164687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:52.165294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.165566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.165832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.166641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:52.166945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:52.168355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:52.186291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 420/840] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Sep 12 11:40:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:52.191801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:52.196186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:52.295531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5159f-5696-436e-a09f-ba43498fe71d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:52.330602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:52.331238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.331518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.331795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.403981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:52.405448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:52.408246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:52.507649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 421/841] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:40:52.513212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:52.594215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:52.621789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e083ce7-f4ef-4d57-90cb-7dc12b90fc6e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:52.655733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:52.656361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.656643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.656913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.667761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:52.669204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-619b8bdf-84ff-4ac0-a856-b9af580de044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:52.683889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:52.684877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:52.685791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:52.685791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:52.685791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:52.685791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 421/842] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:52.692458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:52.802201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce708bf0-d972-4347-924d-cd2554fb43f4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:52.805673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:52.839175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:52.839786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.840068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.840345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.840984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:52.841290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:52.841727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a07df-45f4-4060-b68a-bef9e1d4b7dd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.842020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a07df-45f4-4060-b68a-bef9e1d4b7dd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.842299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a07df-45f4-4060-b68a-bef9e1d4b7dd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.842689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:52.845123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 422/843] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:52.849053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:52.849337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:52.849612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:52.855774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 422/844] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:52.861069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:52.870141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:52.871788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:52.971603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 423/845] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:52.974626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc76bc0-307b-48aa-8d25-025d117ea353 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.013951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.014614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.014950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.015226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.022956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.024455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:53.036348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:53.037224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:53.038269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:53.038736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 423/846] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:53.043968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:53.104331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:53.147130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2f133-6ca7-41ae-80e1-04d5f64ac1df None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.181941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.182569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.182948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.183263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.183962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:53.184359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.185795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.198590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 424/847] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:53.204604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:53.230983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:53.289636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:53.319375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f689-eb6f-45d8-98f8-17a3ceaf167c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.367997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.368835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.369259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.369695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.370617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=091613b4778d4946b0059334bdb70dd1) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:53.371066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.372638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11883e-f1f9-435a-a560-cc895bd260c5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.382453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 424/848] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/role_assignments?user.id=091613b4778d4946b0059334bdb70dd1&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:53.409783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038bad-3257-45c1-81f6-d1e73aeb1735 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.431271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:53.452913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf740-bd4c-4166-baa5-f10dfd822ecf None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.453232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf740-bd4c-4166-baa5-f10dfd822ecf None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.453466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf740-bd4c-4166-baa5-f10dfd822ecf None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.455095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 425/849] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:53.455519 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.456186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.456470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.456744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.457341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.461502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.464701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.465089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.465465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:53.470735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:53.471925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:53.473059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:53.476253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 425/850] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:53.481962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:53.492322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:53.493553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:53.601564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 426/851] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:53.626153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0025ec-9c17-4c4e-9d1a-2c06baf3f178 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.663553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.664159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.664430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.664699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.665326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:53.665624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.667013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f79a9c-a174-47b8-b504-e2582cb02c05 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.672004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 426/852] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:40:53.677478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:53.699018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:53.718126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d39f-e9e9-490d-8b5d-2fc6d133e0cb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.718739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d39f-e9e9-490d-8b5d-2fc6d133e0cb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.719358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d39f-e9e9-490d-8b5d-2fc6d133e0cb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.721973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 427/853] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:53.728493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.729026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.729520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.750804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:53.752574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:53.792206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5f9a9-9d67-4b86-8e85-bcae76103c8b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:53.806791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:53.831962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:53.832583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:53.833196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:53.833468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:53.834414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:53.834751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:53.836394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:53.838628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 428/854] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:53.841050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 427/855] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:53.911475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.032591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed7e57-9345-4e77-b471-9cf58e7df1c1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.069158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.069793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.070071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.070345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.081193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:54.087118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:54.088157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:54.089103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 429/856] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:40:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:54.094271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.110449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.150888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.206203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.228050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638333d3-3bab-4001-9de1-ac42522b9ab9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.238253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14332-60e0-47de-96c7-e17c3516fcce None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.268283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.268907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.269192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.269469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.270145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:54.270448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.272288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.285873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 428/857] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:54.286049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.286449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.288334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.289052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.289332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.289930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.291332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:54.301393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:54.302705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:54.303934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 430/858] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:54.311743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.425051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583e0fe-ccea-4436-ac1f-dfe3b66532f0 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.440598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.444973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d440c-8742-4322-8ca4-dc14178ac639 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.461497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.462060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.462336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.462634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.473428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.475523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.478124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.486414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.487257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.487685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.488112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.488967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:54.489426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.491709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:54.492063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:54.493188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:54.494395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:54.494395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:54.494395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:54.498159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 429/859] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:54.499038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.500114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 431/860] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:54.508789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.600290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371513-688f-460b-8c03-355aefe35043 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.622464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda2404-88f6-4c17-a0ad-eb9fc9466331 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.635948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.636570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.636829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.637096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.637736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:54.638036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.639585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.656928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 430/861] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:54.657803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.662190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.663221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.663682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.664117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.664895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:54.665433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.667169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.678708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 432/862] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:54.723667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.773579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff4ed4-e4e1-4eb5-bc87-8d3f3c81700a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.813921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.814595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.814875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.815145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.824471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.825933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:40:54.843764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:54.845020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:54.846384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 431/863] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:54.851412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:54.884193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:54.958781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11fae-0e5a-4735-b078-cc362cede4a5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:54.994261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:54.995162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:54.995609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:54.996044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:54.996931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:54.997382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:54.998972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:55.017466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 433/864] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:40:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:55.023263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:55.130089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b10c5-7204-4168-a7a9-f6de2f09e97d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:55.148237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:55.168987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:55.169969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.170462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.171071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.242761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:55.244649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:55.348572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 432/865] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:40:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:40:55.354650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:55.447077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:55.464796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8b51-5838-436d-99f3-5466c1dc871a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:55.500891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:55.501447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.501725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.501995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.512996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:55.514433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-316d9878-3ff8-412b-a4df-55b7da8caf14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:55.530372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:55.531423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:55.532963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:55.532963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:55.532963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:55.532963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 434/866] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:55.537440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:55.556069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:55.650953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39e6a2-ebe3-4a0d-bc61-12c6b0dab8b6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:55.687555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd9f7f-b392-42f0-9f1a-265b55517edc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.687890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd9f7f-b392-42f0-9f1a-265b55517edc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.688114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd9f7f-b392-42f0-9f1a-265b55517edc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.689982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 435/867] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:55.694622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:55.695288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606866-dd3e-4208-99ca-b66706610826 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.695572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606866-dd3e-4208-99ca-b66706610826 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.695847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606866-dd3e-4208-99ca-b66706610826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.696227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.696719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.697190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.698055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:55.698727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:55.700444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:55.713635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 433/868] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:55.721072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:55.721233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-44606866-dd3e-4208-99ca-b66706610826 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:55.721233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-44606866-dd3e-4208-99ca-b66706610826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:55.805581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:55.822649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 436/869] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:55.839548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fa9c7-570e-4a82-8c89-bc4fbfcbc931 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:55.879782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:55.880411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:55.880688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:55.881584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:55.888493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:55.889936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:40:55.901931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:55.902890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:55.904178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 434/870] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:40:55.909232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606866-dd3e-4208-99ca-b66706610826 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:55.984205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:56.019181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606866-dd3e-4208-99ca-b66706610826 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.058723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.059256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.059534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.059807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.060664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:56.060974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.062453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d08e-a68f-4ef4-b8e6-573b0639a8da None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.074959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 437/871] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:56.087619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:56.147955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91b411-b890-4a9a-b544-78d5aedc2824 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.148250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91b411-b890-4a9a-b544-78d5aedc2824 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.148532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91b411-b890-4a9a-b544-78d5aedc2824 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.149997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 438/872] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:56.151185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91b411-b890-4a9a-b544-78d5aedc2824 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:56.199213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff18c0-2862-4d36-8646-a11c00624a0e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.235481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.236080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.236354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.237034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.237764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=a17fa3464f414b2da364930144e85fc4) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:56.238070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.240178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef5d7-2817-4927-92f2-4495d851cdb2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.243749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:56.250852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 435/873] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:40:56 2022] GET /identity/v3/role_assignments?user.id=a17fa3464f414b2da364930144e85fc4&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:56.252453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.252739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.253007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.267756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91b411-b890-4a9a-b544-78d5aedc2824 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.279671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:56.280986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:56.307071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:56.320265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.321722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.322474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.322973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.324030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.326433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:56.338951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:56.340035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:40:56.340895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:56.344438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 439/874] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:40:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:56.350439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:56.377558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 436/875] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:56.466352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e15798-194e-47f6-b322-6de8231e2347 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.503028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.503636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.503917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.504189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.504820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:56.505122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.507015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.511804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 440/876] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:40:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:56.517238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:56.626833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffb841-600f-4f94-966e-f8633548ccaf None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.651449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:56.666837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.667706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.667990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.668734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.669630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:56.669954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.670396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:56.672676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff77d7-67ee-4c73-a19a-596d5b6e2a6e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.679785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 437/877] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:40:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:56.722526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:56.776958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676c95-045f-477b-8ffb-86691487d8b8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:56.817428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:56.819105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:56.819640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:56.820120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:56.821008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:56.822658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9470e5-b21f-427b-9dd4-c697d20f01fd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:56.842714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 441/878] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:40:56 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:40:57.078255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be500d-2972-4e37-b798-d7f47a95561c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:57.078757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be500d-2972-4e37-b798-d7f47a95561c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:57.078757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be500d-2972-4e37-b798-d7f47a95561c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:57.080280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 438/879] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:57.086415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c50f7-f5a0-4b08-9f5a-a6c51162f2ce None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:57.086726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c50f7-f5a0-4b08-9f5a-a6c51162f2ce None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:57.086993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c50f7-f5a0-4b08-9f5a-a6c51162f2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:57.105818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0b8c50f7-f5a0-4b08-9f5a-a6c51162f2ce None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:57.107028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0b8c50f7-f5a0-4b08-9f5a-a6c51162f2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:57.192406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 442/880] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:57 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:57.461811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be500d-2972-4e37-b798-d7f47a95561c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:57.565872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be500d-2972-4e37-b798-d7f47a95561c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:57.604051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:57.604682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:57.605324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:57.605602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:57.606284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:57.607717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8c49-2caf-4f31-8016-4fc3a71e30d8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:57.624799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 439/881] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:40:57 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:40:58.658447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d012-abee-48e9-8654-a4132a0125bb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:58.659517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d012-abee-48e9-8654-a4132a0125bb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:58.660011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554d012-abee-48e9-8654-a4132a0125bb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:58.661830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 443/882] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:58.668012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c8777-1e5d-4567-b227-4d25e8df4e95 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:58.668826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c8777-1e5d-4567-b227-4d25e8df4e95 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:58.669319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c8777-1e5d-4567-b227-4d25e8df4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:58.688214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-450c8777-1e5d-4567-b227-4d25e8df4e95 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:58.689875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-450c8777-1e5d-4567-b227-4d25e8df4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:58.741920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:58.766384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 440/883] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:58.924613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae22e-f24f-4fa4-8d5c-31d860af5bcc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:58.925221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae22e-f24f-4fa4-8d5c-31d860af5bcc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:58.925691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae22e-f24f-4fa4-8d5c-31d860af5bcc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:58.927553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 444/884] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:58.933413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674939ea-922a-4cba-b1ab-eccf8d05b501 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:58.934030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674939ea-922a-4cba-b1ab-eccf8d05b501 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:58.934504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674939ea-922a-4cba-b1ab-eccf8d05b501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:58.955153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-674939ea-922a-4cba-b1ab-eccf8d05b501 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:58.956761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-674939ea-922a-4cba-b1ab-eccf8d05b501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:59.028169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 441/885] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:40:59.041913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae22e-f24f-4fa4-8d5c-31d860af5bcc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.062500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f969dc-fec9-46fc-ab33-8c1cc1a3b020 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.062837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f969dc-fec9-46fc-ab33-8c1cc1a3b020 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.063111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f969dc-fec9-46fc-ab33-8c1cc1a3b020 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.065066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 442/886] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:40:59.070639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.070990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.071322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.096074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:40:59.097557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:40:59.102902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.155837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae22e-f24f-4fa4-8d5c-31d860af5bcc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.184801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 443/887] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:40:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:40:59.202989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.203818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.204256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.204682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.216143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:40:59.221153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:59.222271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:40:59.223317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:59.225441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 445/888] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:59.231385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.274928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.329533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.339199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f41bd7-d7f9-475c-bc14-0a4b2378e4c7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.386455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.387910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.388346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.389026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.389837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:59.390525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.392755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.405300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 444/889] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:59.411520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.448309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125fa5c-8a30-4b4f-8959-00b9c7dd7ad5 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.451237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.494147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.494785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.495068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.495343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.496018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.497380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.515570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 446/890] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:40:59 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:40:59.516730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.520243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b2e2a-f032-4a23-b950-27e6c244ebec None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.522705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.584854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.585917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.586676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.586810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.602614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.604291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:40:59.624188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:59.625229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:40:59.626165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:40:59.626165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:40:59.626165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:40:59.630196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 445/891] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:59.631607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.656527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044d7b5-6140-45a1-9008-99eae7a268c2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.697751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.698443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.698443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.698781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.699430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.700757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:40:59.709963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:40:59.711293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:40:59.712191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 447/892] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:40:59.717915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.749345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.775793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c855f7-87aa-4732-8d01-7b89fd849cce None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.818941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.819557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.819898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.820205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.820852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:59.821149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.822643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.834978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264275e-8383-4994-bed6-8877a8ef68be None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.836575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 446/893] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:59.842065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.854925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.873299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.873914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.874192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.874463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.875120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:40:59.875420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:40:59.876765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8526-0b4e-4a6d-b368-db8d63f8cdef None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:40:59.885530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 448/894] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:40:59.885530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c972-39e1-4011-9fa6-b666be1d8b4b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.885530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c972-39e1-4011-9fa6-b666be1d8b4b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.885530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810c972-39e1-4011-9fa6-b666be1d8b4b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.885530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 449/895] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:40:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:40:59.888375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c972-39e1-4011-9fa6-b666be1d8b4b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:40:59.952091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015ec3-e8e3-45f1-b1ba-f64093255c20 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:40:59.986488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:40:59.992652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:40:59.993378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:40:59.993666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:40:59.993948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:40:59.999262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810c972-39e1-4011-9fa6-b666be1d8b4b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.001221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.002712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-8d604104-fa49-4cfc-b952-2859627dd28b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:00.014428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:00.015620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:00.016591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 447/896] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:00.017202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.017202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.017334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.035567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:00.036782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:00.046876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.048024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.048608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.049157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.049939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:00.050556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.053280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.060092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 450/897] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:40:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:00.061281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.102528 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.119566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 448/898] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:00.120655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.190899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a9f1-60fd-4b4f-bae0-abb9f7ed7bf3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.232642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.233586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.234009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.234435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.235359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:00.236016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.237672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.242842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e4e64-e8be-47cb-962b-d206d5ff1dc5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.262006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 451/899] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:00.263178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.279069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.279971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.280660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.281129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.281499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.282432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] RBAC: Adding query filter params (interface=public, service_id=091061c7338d4a46b4882a8e4e7f0424, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:00.282864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.284368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.296175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 449/900] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/endpoints?service_id=091061c7338d4a46b4882a8e4e7f0424&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:00.297764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.372282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.382059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4460-8929-4336-af12-99af596b8189 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.414472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70bff8-46ef-45d9-8e89-c9aef99ad653 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.419242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.420030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.420507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.420918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.452177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.452809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.453139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.453360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.453985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.455451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.463378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 450/901] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:41:00.475818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.506984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.509798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.531761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.595878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda145d-b2f1-4100-abf3-9aee97dcd4f0 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.627908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.639670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.640309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.640583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.641136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.641434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.642826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:00.653261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:00.654263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:00.655768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 451/902] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:00.662717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.665951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 452/903] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:00.672943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.794600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde84a-cb1b-497a-9b8f-7c4f1bab18be None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.804010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60c08e-9318-4670-b33c-638436287054 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:00.822942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.837147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.837853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.838283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.838538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.839270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:00.839649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.841099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.851275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 452/904] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:00.852383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.855601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:00.856445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:00.856731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:00.857003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:00.871495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:00.873150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:00.893078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:00.894104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:00.895066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:00.895066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:00.895066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:00.898611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 453/905] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:00.900192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:00.932375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:00.992556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617132b-0a14-4473-a607-1a87fa2206ae None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.029411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc142b-ed76-4d9c-a299-8039f4980f4e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.038431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.039825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.040327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.041191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.042305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:01.042839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.045178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.057056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 453/906] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:01.058295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:01.079068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.080000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.080529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.081540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.082828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:01.083335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.084941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.103052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 454/907] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:01.111571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:01.147652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:01.194155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6e22-f25d-42be-a3e8-e191eab8e235 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.223395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:01.230274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf553-c1d6-40a5-bdb6-4c8f56239ec3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.244302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.245196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.245642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.246292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.247653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] RBAC: Adding query filter params (interface=public, service_id=b37a391d199d484abe9ff559aae316f1, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:01.248272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.252036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.267328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 454/908] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:41:01 2022] GET /identity/v3/endpoints?service_id=b37a391d199d484abe9ff559aae316f1&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:01.268191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:01.278397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.279070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.279352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.279630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.289076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.290550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:01.305397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:01.306342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:01.307611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:01.310584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 455/909] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:01.312887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:01.421034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:01.421588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60637a4c-0f26-4ee2-9739-9d6e6c23c85f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.441774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ae87-5e01-4f46-ae31-619912e71de8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.471232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.472931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.473441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.474337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.475649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.477596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.480035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.480634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.480976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.481359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.482135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:01.482414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.483875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943e6e0-41a3-4841-a7fc-b40f2c85ada2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.486998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 455/910] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:01 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:41:01.506652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 456/911] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:01.509200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:01.567698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:01.625698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bbcfc-17da-4e0c-8e98-b0aba1e332a7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:01.674006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:01.674662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.674946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.675219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.675961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] RBAC: Adding query filter params (role.id=44ed3e7aa7f3406398cf78e3f4f8c866, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=a17fa3464f414b2da364930144e85fc4) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:01.676270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:01.678579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165967a-3b72-4980-8bab-9afff02fb691 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:01.685151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 456/912] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:01 2022] GET /identity/v3/role_assignments?user.id=a17fa3464f414b2da364930144e85fc4&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=44ed3e7aa7f3406398cf78e3f4f8c866 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:01.774599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244f943-a5a2-4443-8f5a-a604aebe39e8 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.775269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244f943-a5a2-4443-8f5a-a604aebe39e8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.775764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244f943-a5a2-4443-8f5a-a604aebe39e8 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.777695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 457/913] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:01.783913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:01.784590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:01.785075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:01.805770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:01.807924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:01.885722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 457/914] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:02.165988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244f943-a5a2-4443-8f5a-a604aebe39e8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:02.277424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244f943-a5a2-4443-8f5a-a604aebe39e8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:02.314804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:02.315695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.316154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.316644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.317385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:02.319694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:02.329238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:02.330457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:02.331402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:02.332614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 458/915] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:02.340084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:02.385080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:02.455314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee95a6f-515f-4f5c-b26e-25bec7f14248 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:02.495189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:02.495796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.496073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.496341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.496967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:02.497265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:02.498653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:02.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 458/916] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:02.511628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:02.523595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:02.626907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43ea93-4eac-4a0a-84df-2fb76a5a7093 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:02.664970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:02.665583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.665867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.666142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.666807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:02.667114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:02.668485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec531e-36a0-42c3-bffc-54bb924936bb None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:02.673877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 459/917] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:02.680929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:02.728271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e323e7-0a0b-4606-9663-2e49bf80f729 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.728570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e323e7-0a0b-4606-9663-2e49bf80f729 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.728754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e323e7-0a0b-4606-9663-2e49bf80f729 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.730228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 460/918] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:02.735300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.735694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.736070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.757792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:02.759278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:02.791324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a687fb-0bc8-4b11-b70c-2646b9714d86 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:02.799727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:02.832901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:02.833736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.834503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.834830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.836120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] RBAC: Adding query filter params (interface=public, service_id=9b585013b05b4bd0a69e35c495b8b505, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:02.836562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:02.838396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:02.848384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 461/920] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:02.848741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 459/920] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:41:02 2022] GET /identity/v3/endpoints?service_id=9b585013b05b4bd0a69e35c495b8b505&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:02.849782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:02.852253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:02.957437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfc3fa-6944-4a25-b78a-968720775802 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:02.995675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:02.996606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:02.997114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:02.997610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:02.998420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:03.000076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:03.007697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 462/921] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:02 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:41:03.152607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:03.259687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106aa8-8a25-4287-97a2-6d63522a98df None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:03.296197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:03.296841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.297125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.297400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.298046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:03.299465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:03.309433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:03.310419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 11:41:03.311423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:03.314620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 460/922] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Sep 12 11:41:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:03.319667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:03.390205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:03.429088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87e106-16d5-4712-a654-b816e6cefbe4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:03.466801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:03.467417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.467698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.467974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.468611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:03.468927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:03.470358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:03.480648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 463/923] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:03.482008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:03.518430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:03.589684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebf63d-46b0-41f3-9a6d-4a3aa004e41e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:03.625906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:03.626553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.626837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.627117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.627747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:03.628131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:03.629515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd601520-3222-4ba6-9877-0081d8e9899c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:03.640356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 461/924] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:03.641733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:03.755562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329f919-eef1-40f2-82f7-5f2a52549918 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:03.780123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaec4f-bafb-470a-b52e-57f7c7806206 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.780298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaec4f-bafb-470a-b52e-57f7c7806206 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.780580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaec4f-bafb-470a-b52e-57f7c7806206 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.782144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 462/925] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:03.787277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3dc4d-d102-4b0a-90b9-20b93184f50c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.787591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3dc4d-d102-4b0a-90b9-20b93184f50c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.787830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3dc4d-d102-4b0a-90b9-20b93184f50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.789166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:03.796812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:03.797850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.798371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.798887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.799789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:03.801361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92035e49-bfa0-420e-8480-254ab1c170e5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:03.808553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-13c3dc4d-d102-4b0a-90b9-20b93184f50c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:03.809680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-13c3dc4d-d102-4b0a-90b9-20b93184f50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:03.829972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 464/926] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:03.916922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 463/927] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:03.957618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2f41-1e50-42cf-abee-152a99212316 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.957899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2f41-1e50-42cf-abee-152a99212316 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.958179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2f41-1e50-42cf-abee-152a99212316 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.960270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 465/928] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:03.966009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:03.966566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:03.967031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:03.989335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:03.991371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:04.012795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:04.072450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 464/929] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:04.184690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2f41-1e50-42cf-abee-152a99212316 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.292366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2f41-1e50-42cf-abee-152a99212316 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.329510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.330137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.330374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.330668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.331261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.332631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:04.342921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:04.343940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 11:41:04.344792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:04.345533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.345615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 466/930] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:04.352741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.425525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:04.464730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ed70f-d014-4651-a901-1a46c5349eba None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.493933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf459a-5e1b-4c04-88f0-00e6a9cc05b6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.503659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.504672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.505181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.506076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.517830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:04.524253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:04.525220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:04.526103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:04.528975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 465/931] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:04.534590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.538835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.539913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.540389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.541785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.542605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:04.543050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.544560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.551590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 467/932] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:04.556588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:04.557810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.668481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe762b1-5adf-42c2-ac2c-d6772164a20a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.669306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90961c0-3b96-4da3-b2ac-054f5b01105a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.688444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:04.712426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.713616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.714105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.714214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.714755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.714863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.715110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.715384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.716013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:04.716316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.716605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:04.717104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.717662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.719015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.728859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 468/933] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:04.731153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 466/934] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:04.731690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.737257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.836420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:04.846533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69951-857e-44a7-b807-93861c717d9d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.869301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd31ca-c95f-4532-9f5c-8b786701691c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:04.886577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.887619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.888128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.888691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.889695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.891355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.909021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:04.909701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:04.909975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:04.910279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:04.927730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 469/935] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:04.929945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:04.931488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:04.950984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:04.952111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:04.953067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:04.953067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:04.953067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:04.953067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 467/936] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:04.958629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:04.999795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:05.082418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29622d-67e2-4a6b-947d-06631f53a281 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:05.131305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:05.131948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.132228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.132500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.133164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:05.133537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:05.135002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:05.149967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 470/937] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:05.155634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.266299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab1628-f126-4265-8d37-cdbbabe140eb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:05.294708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:05.304679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:05.305325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.305609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.305884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.314896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:05.316403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-37a38e2b-0584-4f45-ae5f-98e6c9f89e14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:05.329287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:05.330338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:05.331695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 468/938] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:05.339686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.436875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b19c-123c-43f3-924c-9742a2521323 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.437201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b19c-123c-43f3-924c-9742a2521323 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.437431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b19c-123c-43f3-924c-9742a2521323 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.439076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 469/939] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:05.444282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:05.445436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.445617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.445908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.458758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6acd9-33dc-482a-9de3-ce1e15f73f94 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:05.469924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:05.471551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:05.502695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:05.503921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.504622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.505664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.507416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:05.508115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:05.510641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:05.538883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 471/940] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:05.539694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.559148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 470/941] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:05.660738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b0099-ff28-45d3-a862-9f89299962f4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:05.699451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:05.700195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.700527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.700806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.712325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:05.713772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:05.722684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:05.730386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:05.731513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:05.732609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:05.732609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:05.732609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:05.734531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 472/942] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:05.740904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.829376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.849505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcb808-d70b-4eb7-848e-c32dfdae2bd2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:05.873562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:05.895022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:05.896364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:05.896813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:05.897281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:05.898024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:05.898748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:05.900304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:05.920327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 471/943] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:05.921568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:05.969746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad38d-4142-460b-8d70-2c4c330adfec None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.017435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.018349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.019633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.020073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.021478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.023781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:06.038251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:06.039422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:06.040270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:06.043399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 473/944] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:06.049109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.054227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a781-ceb1-4522-b37a-135855921323 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.097548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.098448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.099144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.099600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.107258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.109028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:06.122777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:06.123655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:06.124551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:06.131604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 472/945] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:06.132372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.133909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.167175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb2092-1b98-4166-867a-fcc14ecf5c60 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.173614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.204236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.204841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.205120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.205391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.206016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:06.206317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.207778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1500a-7088-4d31-9d06-1b3d296fb988 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.218558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 474/946] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:06.219758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7331a-ed3a-4a71-ab23-46efa0fc4950 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.220045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7331a-ed3a-4a71-ab23-46efa0fc4950 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.220321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7331a-ed3a-4a71-ab23-46efa0fc4950 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.221726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 475/947] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:06.222415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7331a-ed3a-4a71-ab23-46efa0fc4950 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.237978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4598e0d-1d0e-4e83-9493-568ea1380a72 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.270586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.271498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.271782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.272053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.272753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:06.273040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.274552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b786f4-9283-4367-8ecc-ebf89b0ced54 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.287538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 473/948] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:06.288834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.289116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.289387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.307808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:06.309242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:06.334820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7331a-ed3a-4a71-ab23-46efa0fc4950 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.360724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.374227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.374867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.375144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.375670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.376043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:06.376342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.377689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.381313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 474/949] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:06.382315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.390856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 476/950] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:06.391628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.419435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.503570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2d6aa-151d-4636-bcfb-fd20feec3030 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.511656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851ab5-7b3b-47be-a232-adeb87734073 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.539638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.540308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.540591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.540865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/a17fa3464f414b2da364930144e85fc4/roles/44ed3e7aa7f3406398cf78e3f4f8c866` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.543061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=a17fa3464f414b2da364930144e85fc4, role_id=44ed3e7aa7f3406398cf78e3f4f8c866)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.544934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.546742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.547344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.547618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.547890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.548632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.550051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.562099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 475/951] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:41:06 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/a17fa3464f414b2da364930144e85fc4/roles/44ed3e7aa7f3406398cf78e3f4f8c866 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:41:06.580164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 477/952] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:41:06.661356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.779031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.795311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78d525-bdc2-44c9-8df3-2013aa1f5132 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.836996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.837643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.837903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.838173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.838794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:06.840156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:06.849435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:06.850454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:06.851333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 476/953] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:06.856702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:06.908016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:06.962753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c644e6-0419-44cf-a035-7a4acd79e358 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:06.997825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:06.998439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:06.998742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:06.999011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:06.999643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:06.999945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:07.001302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ff474-8651-4a03-8523-a4e85467c824 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:07.007215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 478/954] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:07.012814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:07.115471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78709c-c1f7-4ace-b5bc-fd8e007bbd58 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:07.151314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:07.151982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:07.152258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:07.152530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:07.153262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:07.153571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:07.154977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf05ba1-c7fc-41ff-b876-be28be9111f2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:07.159893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 477/955] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:07.377957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a95e-cde7-40e8-af43-a39b83bf54dd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:07.378472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a95e-cde7-40e8-af43-a39b83bf54dd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:07.378621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444a95e-cde7-40e8-af43-a39b83bf54dd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:07.380687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 479/956] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:07.386740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:07.387029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:07.387308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:07.406261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:07.407494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:07.415553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:07.484734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 478/957] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:07.754539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444a95e-cde7-40e8-af43-a39b83bf54dd None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:07.859469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444a95e-cde7-40e8-af43-a39b83bf54dd None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:07.873586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:07.895704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:07.896832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:07.897102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:07.897438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:07.898102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:07.899552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:07.902163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING keystone.server.flask.common [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 11:41:07.923162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.sql.core [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 11:41:07.923162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 11:41:07.923162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [parameters: {'id': '5a76340af1834da5ac700bdeedbb2f62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 11:41:07.923162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81387) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a76340af1834da5ac700bdeedbb2f62.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a76340af1834da5ac700bdeedbb2f62. Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:41:07.940173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 12 11:41:07.941063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application yield resource Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise self.value Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application session.commit() Sep 12 11:41:07.942194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.session.flush() Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application compat.raise_( Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 11:41:07.944183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 11:41:07.945143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application result.read() Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:41:07.945988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [parameters: {'id': '5a76340af1834da5ac700bdeedbb2f62', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:07.952121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:07.953176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5a76340af1834da5ac700bdeedbb2f62. Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:07.954366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 480/958] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Sep 12 11:41:07 2022] POST /identity/v3/projects => generated 262 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:07.955124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:08.058463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839eb94-e3e1-4fcc-9ea6-9454b83e887f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:08.094449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:08.095121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.095400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.095675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.103804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:08.105224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-9f5cc68e-78b7-4d0e-bf3a-ebd9174fcddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:08.118945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:08.121292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:08.122987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 479/959] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:08.126382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:08.237648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed5773-3d7a-4495-838e-c200cbf36e48 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:08.272307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:08.272909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.273188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.273463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.274131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:08.274436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:08.276097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748039-ac91-49ab-9e96-3b87b288be39 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:08.296738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 481/960] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:08.911429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2539b-d254-4d04-8774-ff2dbea9d3de None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.911944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2539b-d254-4d04-8774-ff2dbea9d3de None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.911944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2539b-d254-4d04-8774-ff2dbea9d3de None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.913679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 480/961] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:08.919020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.919559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.919559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.939414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:08.940909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:08.952027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6012-185f-45c5-a937-5781a834d48b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.952661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6012-185f-45c5-a937-5781a834d48b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.952940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be6012-185f-45c5-a937-5781a834d48b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.954958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 481/962] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:08.960943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:08.961226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:08.961502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:08.967765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:08.968594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:08.985736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:08.987959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:09.027439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 482/963] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:09.074240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 482/964] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:09.298744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.353803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.405632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114fc63-5cd0-4b9c-9ee8-fb301efb4296 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.442042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00d693-7636-4224-9344-1084879bdaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.443318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d693-7636-4224-9344-1084879bdaab None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.443756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d693-7636-4224-9344-1084879bdaab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.444033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d693-7636-4224-9344-1084879bdaab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.444667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00d693-7636-4224-9344-1084879bdaab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.445991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00d693-7636-4224-9344-1084879bdaab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-be00d693-7636-4224-9344-1084879bdaab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:09.454955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:09.456039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:09.456889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 483/965] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:09.459075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f58e5-8c3e-43ff-af45-47de2fd65b28 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.459552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f58e5-8c3e-43ff-af45-47de2fd65b28 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.460102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f58e5-8c3e-43ff-af45-47de2fd65b28 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.461640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 484/966] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:09.463346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f58e5-8c3e-43ff-af45-47de2fd65b28 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.471850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8b4e-27ff-4e76-8aac-10ad038314d8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.487657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:09.508142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:09.514376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.515143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.515845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.515845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.527072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1a0bf9b5-bda6-4f14-9e25-b3d8aafce3f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:09.531922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:09.532883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:09.533738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 483/967] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:09.535279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.535279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.535279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.553621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:09.554746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:09.580872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f58e5-8c3e-43ff-af45-47de2fd65b28 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.617650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.618264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.618565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.618841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.619473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:09.619774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.621127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.626557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 485/968] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:09.627431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.628756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 484/969] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:09.635705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.736444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:09.754135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4b84e-73e5-462a-bce4-5dde9153203d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.779490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4b335-7536-4751-b848-19c894f346dc None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.791430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.792299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.792774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.793257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.794127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:09.794657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.796188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.804684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:09.805446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 486/970] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:09.815646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.821108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.821726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.822008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.822281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.822942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:09.823246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.824634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.829978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 485/971] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:09.836430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:09.938404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605fb4-fbaf-4ff9-8657-97738b4ec43a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.946324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00064-d3c7-45a4-a16b-09a9822ec13f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:09.975968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.976606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.976890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.977167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.983062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:09.983678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:09.983961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:09.984236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:09.984945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] RBAC: Adding query filter params (interface=admin, service_id=9b585013b05b4bd0a69e35c495b8b505, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:09.985253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.986653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.988204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:09.989717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:09.996687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 486/972] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/endpoints?service_id=9b585013b05b4bd0a69e35c495b8b505&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:09.996687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.003820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:10.011228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:10.012300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:10.013226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:10.013226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:10.013226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:10.015743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 487/973] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:10.021740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.053287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:10.119653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70f5-ce82-4169-9fcd-d88158a7b501 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.135274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c185-8cb7-4e94-ab8e-0dcc92b7b339 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.156031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.156642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.156921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.157199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.157851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.159250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.172744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.173358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.173642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.173919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.178758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 487/974] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:41:09 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:10.178846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.178920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.178991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.193211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 488/975] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:10 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:41:10.314767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f2ccc-cf2d-464e-9624-0bc8f73bebcd None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.328204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:10.358882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.359537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.359776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.360047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.360709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:10.361011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.362443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.376989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 488/976] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:10.384549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.459534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:10.506106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011567e-d3ce-4a83-9c20-99ca8f257d17 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.542236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.542899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.543177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.543447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.551480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.553015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-a09ca20d-1759-4f1b-8add-f973c39f5bd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:10.564821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:10.565816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:10.566912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 489/977] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:41:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:10.572105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.593717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec90-eba4-4ffb-9f20-eb43c1f91e95 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.594019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec90-eba4-4ffb-9f20-eb43c1f91e95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.594275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318ec90-eba4-4ffb-9f20-eb43c1f91e95 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.595919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 490/978] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:10.601411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.601688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.601959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.621161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:10.622345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:10.681431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1532d-f976-4a0e-a091-3e4cbf340d4b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.702133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 491/979] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:10.721175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.721797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.722073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.722407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.723179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:10.724010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.725642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.743952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 489/980] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Sep 12 11:41:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:10.749556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:10.808747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:10.856132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d676fb-2573-45d2-b648-92537258dc3f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:10.891312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:10.892098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:10.892483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:10.892856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:10.962779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:10.964317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:10.978023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.012251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.026399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.089232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 492/981] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:10 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:11.098650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.107255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6057e66-f120-4d4e-9129-ac8bfa0ef65c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.154179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.155225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.156015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.156737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.157551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.159645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.163577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING keystone.server.flask.common [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 11:41:11.181199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.sql.core [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 11:41:11.181199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 11:41:11.181199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [parameters: {'user_id': '35e09df404db4ee9929a1616081e1d83', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 11:41:11.181199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81386) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result.read() Sep 12 11:41:11.188399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:41:11.189349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application yield resource Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application next(self.gen) Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise self.value Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application session.commit() Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 11:41:11.190722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.session.flush() Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application compat.raise_( Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 11:41:11.191860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 11:41:11.193297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application conn.query(q) Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application result.read() Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 11:41:11.194499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [parameters: {'user_id': '35e09df404db4ee9929a1616081e1d83', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:11.195552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 12 11:41:11.196559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.197710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 490/982] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Sep 12 11:41:10 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:11.199073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.236608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23398-cd81-4530-9da6-2d2d4acb5c86 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.289320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.290332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.291037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.291466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.292571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.309246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.311920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.313874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0cec5-9050-4257-9537-55ce487beed8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:11.331563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:11.333265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:11.334614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:11.334614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:11.334614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:11.337247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 493/983] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:11.342755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.356098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.359095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.359799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.360075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.360343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.369916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.371753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:11.384597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:11.385573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:11.386944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 491/984] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:11.391642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.460826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb367407-38c6-409e-be0f-b62c110aab47 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.497132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.497819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.498195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.498603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.499342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:11.499772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.501223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.518785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 494/985] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:11.520210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.524927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74144c-b1bf-4787-adb5-66f84fbb70c5 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.571960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.573112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.574050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.574485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.575664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:11.576121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.578351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.594777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.624453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 492/986] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:11.647694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e199b-0484-4622-aebd-6be30ff6d8ea None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.690635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.691559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.692053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.692774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.701207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.702940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:11.718562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:11.719742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:11.720877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:11.723185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 495/987] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:11.731436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:11.738949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:11.848829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04e41-efe6-4260-9d99-8890b0d470fe None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:11.884614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:11.885233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:11.885504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:11.885775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:11.886501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:11.886800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:11.888292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7ee29-34f6-4bc6-a00d-611541fedd57 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:11.901984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 493/988] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:11.907795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:12.017324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454444c-b097-4d1c-ace0-81cb83c9a54c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:12.044371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:12.054299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:12.054950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.055233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.055507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.056239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] RBAC: Adding query filter params (role.id=44ed3e7aa7f3406398cf78e3f4f8c866, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=a17fa3464f414b2da364930144e85fc4) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:12.056552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:12.057971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050d6d7-b541-401e-b000-f0b797e1d9f6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:12.070575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 496/989] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:11 2022] GET /identity/v3/role_assignments?user.id=a17fa3464f414b2da364930144e85fc4&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=44ed3e7aa7f3406398cf78e3f4f8c866 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:12.426928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae55-2897-41ae-8b29-6058ac181fd1 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.427437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae55-2897-41ae-8b29-6058ac181fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.427517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae55-2897-41ae-8b29-6058ac181fd1 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.429158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 494/990] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:12.434867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55db59-125e-48c6-a760-d6af0fd0258e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.435161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55db59-125e-48c6-a760-d6af0fd0258e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.435419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55db59-125e-48c6-a760-d6af0fd0258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.454642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2d55db59-125e-48c6-a760-d6af0fd0258e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:12.455826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d55db59-125e-48c6-a760-d6af0fd0258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:12.525134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 497/991] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:12.536260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700f45-012b-475b-9211-d78a748ca582 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.536620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700f45-012b-475b-9211-d78a748ca582 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.536970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700f45-012b-475b-9211-d78a748ca582 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.538399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 495/992] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:12.543679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.543975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.544256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.563059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:12.564245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:12.634319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 498/993] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:12.791981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700f45-012b-475b-9211-d78a748ca582 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:12.856684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:12.896566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700f45-012b-475b-9211-d78a748ca582 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:12.903543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:12.931681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:12.932314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:12.932593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:12.932866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:12.933444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:12.934853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:12.943792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:12.944806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:12.946319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 496/994] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Sep 12 11:41:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:12.950590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:12.958890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.012578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc7603-0e16-43ca-960d-11c6e9d1cae4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.048392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.048989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.049289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.049498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.050085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.051641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.055514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996ff0b-8169-4224-ae4b-bba47cfc0f5d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:13.060641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:13.061630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:13.062537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:13.066743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 499/995] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:13.067909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.089554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.099755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89832559-112a-4715-894f-5460173c2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.100566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89832559-112a-4715-894f-5460173c2992 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.100934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89832559-112a-4715-894f-5460173c2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.101151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89832559-112a-4715-894f-5460173c2992 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.101741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89832559-112a-4715-894f-5460173c2992 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:13.102040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89832559-112a-4715-894f-5460173c2992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.103688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89832559-112a-4715-894f-5460173c2992 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.113636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 497/996] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:13.116483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89832559-112a-4715-894f-5460173c2992 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.184402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010089-7e73-46da-b6c8-fabf3ff60ffd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.203380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.225722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89832559-112a-4715-894f-5460173c2992 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.231172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.232023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.232484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.232947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.234246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:13.234774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.237206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.245245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 500/997] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:13.250434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.262094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.262741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.263023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.263295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.263924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:13.264226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.265591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.271164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 498/998] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Sep 12 11:41:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:13.276564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.333697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.370401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7238c9-d9c1-4174-b6a7-67979f01cadc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.378297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c949e44-7146-4c87-b9ef-c531744da662 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.410076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.411233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.411647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.412060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.413022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:13.413484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.415671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.417952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.418603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.418885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.419163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.419861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.421220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.423852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 501/999] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:13.429771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.445983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 499/1000] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:41:13.482316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.549247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dd4a8-3df5-4dc9-aaa8-8f6164bf2992 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.592647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.593272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.593550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.593825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.594551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] RBAC: Adding query filter params (interface=public, service_id=15eb2e6d92984ca7a45b6ac9f3c31998, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:13.594864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.596266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eda39-f236-4326-84e9-2e12ae93abd6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.606706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 502/1001] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:41:13 2022] GET /identity/v3/endpoints?service_id=15eb2e6d92984ca7a45b6ac9f3c31998&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:13.608366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:13.717629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afd948-9b8b-4d91-8016-73e73be89fd4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:13.737355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:13.760147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:13.761001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:13.761455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:13.761878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:13.762671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:13.764178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99fc1b-c799-4236-8c81-e617dbf8e455 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:13.778539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 500/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:13 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:41:14.254276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f663-d6c3-4932-a211-eb4fcce59023 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:14.255378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f663-d6c3-4932-a211-eb4fcce59023 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:14.255460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f663-d6c3-4932-a211-eb4fcce59023 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:14.256817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 503/1003] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:14.262057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:14.262344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:14.262650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:14.281211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:14.282749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:14.351929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 501/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:14.618650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f663-d6c3-4932-a211-eb4fcce59023 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:14.724541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f663-d6c3-4932-a211-eb4fcce59023 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:14.726059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:14.760366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:14.760993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:14.761272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:14.761544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] PATH_INFO: `/v3/services/image` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:14.762124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:14.763501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:14.774984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:14.776058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:14.776936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:14.778575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 504/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 12 11:41:14 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:14.782675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:14.881785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1436b9e-7071-477f-a941-45426d0a31a6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:14.892248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:14.917633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:14.918240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:14.918539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:14.918817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:14.919447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] RBAC: Adding query filter params (name=image) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:14.919750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:14.921117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861a1d8-2981-4b45-be03-1f3f64c027de None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:14.930686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 502/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:14.932144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:15.033864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3bc00-3e6d-467c-8789-3bc9e0dbbd53 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:15.074246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:15.074888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:15.075168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:15.075442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:15.076072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] RBAC: Adding query filter params (type=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:15.076375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:15.077737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039980d-eb71-4461-96d6-291d3b984d76 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:15.090557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 505/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:15.629052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa495-0730-4f2c-b25f-302712093b9c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:15.629505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa495-0730-4f2c-b25f-302712093b9c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:15.629770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa495-0730-4f2c-b25f-302712093b9c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:15.631427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 503/1008] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:15.636480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023828-1893-45ae-807d-2ac65a58fb25 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:15.636939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023828-1893-45ae-807d-2ac65a58fb25 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:15.637018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd023828-1893-45ae-807d-2ac65a58fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:15.655798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dd023828-1893-45ae-807d-2ac65a58fb25 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:15.656970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dd023828-1893-45ae-807d-2ac65a58fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:15.682901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:15.729607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 506/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:15.944344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe79e5-5f31-4b3f-9934-05bb80c8886b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:15.944851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe79e5-5f31-4b3f-9934-05bb80c8886b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:15.944851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe79e5-5f31-4b3f-9934-05bb80c8886b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:15.946264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 504/1010] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:15.951334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:15.951628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:15.951904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:15.970916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:15.972149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:16.002639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe79e5-5f31-4b3f-9934-05bb80c8886b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.045144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 507/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:16.107989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe79e5-5f31-4b3f-9934-05bb80c8886b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.143682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.144304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.144581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.144848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.146051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.147496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.149210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 505/1012] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:16.154684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.244789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:16.256212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2accb-acb2-4ab1-bbee-b33ff8c53a95 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.290491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.291117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.291393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.291661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.292313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.293724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.312938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.326345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 508/1013] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:41:16 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:16.363029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:16.425277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3648-8404-47d1-b653-f19d79d8a4b5 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.462972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.463617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.463894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.464168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.464757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.466179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:16.475864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:16.476946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:16.479692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 506/1014] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:16.483217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.596761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c067-a549-46dc-86e9-07b383f30eac None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.632275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.632885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.633206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.633482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.634108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:16.634408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.635937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.646340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 509/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:16.647570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.759092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8c56-bb56-458a-ad83-e9949b532343 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.766253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:16.795269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.795891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.796175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.796452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.797150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:16.797463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.798924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bbbc5-03a9-4cc6-b32d-2bbab067e096 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.808693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 507/1016] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Sep 12 11:41:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:16.809812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:16.820454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:16.918381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23a71-7f2e-4750-91b8-18c05691aa4d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:16.954165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:16.954818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:16.955096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:16.955367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:16.956035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:16.957509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54452e2b-44a9-4b51-80dc-d2a65231af24 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:16.986545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 510/1017] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:17.273813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76cde-7581-4cac-aa81-0fb60643a992 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:17.274410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76cde-7581-4cac-aa81-0fb60643a992 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:17.274410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76cde-7581-4cac-aa81-0fb60643a992 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:17.275708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 508/1018] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:17.281950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1734-f56c-47b5-9aa6-74c20b8b118a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:17.282241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1734-f56c-47b5-9aa6-74c20b8b118a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:17.282503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1734-f56c-47b5-9aa6-74c20b8b118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:17.301615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3e5c1734-f56c-47b5-9aa6-74c20b8b118a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:17.302984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3e5c1734-f56c-47b5-9aa6-74c20b8b118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:17.361464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:17.380159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 511/1019] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:17 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:17.648966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf76cde-7581-4cac-aa81-0fb60643a992 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:17.753544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf76cde-7581-4cac-aa81-0fb60643a992 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:17.788155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:17.788769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:17.789054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:17.789318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:17.789980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:17.791387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf7d24-5b8a-4738-915d-44f3265ad3ee None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:17.807886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 509/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:41:17 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:18.562365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80c0e4-a9e6-4bf3-abad-257fc51510eb None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:18.562922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80c0e4-a9e6-4bf3-abad-257fc51510eb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:18.562922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80c0e4-a9e6-4bf3-abad-257fc51510eb None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:18.564356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 512/1021] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:18.569567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:18.569952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:18.570200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:18.588887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:18.590048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:18.614940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:18.666233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 510/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:18.937675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80c0e4-a9e6-4bf3-abad-257fc51510eb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:19.044054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80c0e4-a9e6-4bf3-abad-257fc51510eb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:19.078970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:19.079603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.079880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.080152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.081269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:19.086559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659eb21-750a-4921-b939-fe7ae2ce0190 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:19.086559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 513/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:19.091447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:19.196662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c41d-e4a9-4b77-a54e-feb81ef0771c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:19.231092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:19.231701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.231978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.232251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.232904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:19.234337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:19.245712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5a4e-d0cf-4589-9cbc-851a524da649 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.246038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5a4e-d0cf-4589-9cbc-851a524da649 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.246412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5a4e-d0cf-4589-9cbc-851a524da649 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.248000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 514/1024] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:19.253521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.253896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.254784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.262580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 511/1025] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:41:19 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:41:19.276981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:19.278216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:19.340155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:19.359757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 515/1026] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:19.632619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:19.648880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:19.736262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f801e2-0946-4bcf-8936-f83ccf131701 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:19.769472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:19.770738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.771017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.771284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.781207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:19.787162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:19.788100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:19.788979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:19.790594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 512/1027] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:41:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:19.794485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:19.897763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5d30c-6bf3-429b-b27c-aafbb61b9f34 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:19.934801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:19.935408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:19.935686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:19.935956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:19.936614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:19.936916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:19.938273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ee318-b155-4737-9a42-276fea57e6e0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:19.951395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 516/1028] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:41:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:19.952584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.001887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab401f1-a15b-4715-b02c-f3581874c704 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.002219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab401f1-a15b-4715-b02c-f3581874c704 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.002437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab401f1-a15b-4715-b02c-f3581874c704 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.003834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 517/1029] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:20.009614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.010091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.010502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.024973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.029647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:20.031092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:20.061822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a831a2-6d10-4d9f-8867-ceab5ee65b4d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.077932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.100342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.100974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.101251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.101738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.106549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 518/1030] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:20.118063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.119556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:20.134156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:20.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:20.140630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:20.140630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:20.140630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:20.140630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 513/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:20.141251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.246040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03ba89-ccd1-483e-b459-8cfc8048dc2b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.280346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.280952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.281234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.281517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.282157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:20.282461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.283857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.299442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 519/1032] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:20.300555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.396565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.402091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694d554-195e-4444-bbfa-bf0c9ee9eaf9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.426546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.439703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.440223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.441015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.441349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.441850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.451375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.452823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:20.465039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:20.466106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:20.467109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 514/1033] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:20.471679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.508727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7411c-8c88-4145-a3e7-e3619026e5ae None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.543489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.544119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.544394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.544664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] PATH_INFO: `/v3/services/image` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.545242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.546623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:20.555628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:20.556639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:20.557501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 520/1034] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:20.562422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.582836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd187-06a2-419f-b7a7-7c49f6593adc None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.618430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.619077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.619359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.619629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.620361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:20.620667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.622094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.644453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 515/1035] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:20.646539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.677331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a4fd4-7563-4564-a5cd-12997928f01b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.679387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.693829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.714581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.715191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.715472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.715745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.716367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:20.716663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.718016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.723424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 521/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:20.728810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.764577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507d47-c9f2-4b12-8449-a31a27822ea9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.801840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.802678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.803102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.803518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.829656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5016e-47cc-493a-9ef4-07c231029ff0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:20.870089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:20.870717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:20.870994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:20.871260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:20.871880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] RBAC: Adding query filter params (type=image) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:20.872175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.873526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.878825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 522/1037] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:20.884415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:20.886194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:20.887692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:20.917577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.933947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:20.994148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a26b2-64c0-4d14-9730-832b40351b78 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.000193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 516/1038] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:21.001196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.033977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.034492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.034795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.035064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.037034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] RBAC: Adding query filter params (interface=public, service_id=4081a314cde24c2ba13cd1b777ef5613, region_id=RegionOne) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:21.037034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.037980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.043292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 523/1039] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:41:20 2022] GET /identity/v3/endpoints?service_id=4081a314cde24c2ba13cd1b777ef5613&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:21.048650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.107974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856526d5-040d-493f-8d7e-2642f1b076f0 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.148401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.149029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.149303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.149571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.158875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:21.160018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.162243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.164237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:21.167716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb293e-3aec-4882-a6aa-320e3ddc5a91 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:21.179511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:21.180490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:21.181397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:21.181397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:21.181397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:21.185345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 517/1040] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:21.186172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.204628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.205244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.205634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.205908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.206536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.208575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.226559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 524/1041] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:41:21.308544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789dc5-0126-4b0c-b332-a2f4c0a4a92f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.344145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.344777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.345052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.345322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.345977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:21.346276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.348266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c719-1ac4-4957-9f41-b47810754f61 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.366349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 518/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:21.367703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.478287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc17c1-05cc-49fc-b303-ba7523694c0f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.498645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bcda-b132-4d28-af6c-c1dd4b7ed980 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.498940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bcda-b132-4d28-af6c-c1dd4b7ed980 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.499209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bcda-b132-4d28-af6c-c1dd4b7ed980 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.500785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 519/1043] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:21.506264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.506568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.506841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.515309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.515942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.516216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.516485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.524275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.525225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:21.525704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.526414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:21.537483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:21.538444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:21.539691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 525/1044] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:21.544225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.564548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:21.582237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:21.603142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 520/1045] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:21.654775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4fc3-4889-42f6-aa96-9b3328e6e061 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.690127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.690776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.691063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.691341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.692508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:21.692823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.694248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.716465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 526/1046] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:21.718084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.822384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646fc06-597f-4592-922a-7ddc58d824f0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:21.857060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:21.857668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:21.857948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:21.858218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:21.858989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] RBAC: Adding query filter params (role.id=44ed3e7aa7f3406398cf78e3f4f8c866, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:21.859300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:21.860736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:21.873906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 521/1047] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=44ed3e7aa7f3406398cf78e3f4f8c866 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:21.875384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:21.961708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:21.986182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76117-2417-439e-82a0-73d96b0a0630 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.026443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.027539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.028032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.028502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.041687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:22.047132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:22.048125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:22.049033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:22.050653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 527/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:22.056507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:22.123531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:22.175626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1517de-947a-4593-8e4a-2a0ae147c106 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.210310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.210974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.211253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.211523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.212188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:22.212490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:22.213909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:22.227525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 522/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:22.228680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:22.330242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed8ed8-438a-4d42-a431-11c2020ea5eb None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.366818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.367371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.367648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.367921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] PATH_INFO: `/v3/users/6d30d531408d49bd98102be9777e643c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.375861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:22.383581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] RBAC: Authorizing `identity:get_user(user_id=6d30d531408d49bd98102be9777e643c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:22.385147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:22.391342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 528/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:22 2022] GET /identity/v3/users/6d30d531408d49bd98102be9777e643c => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:22.392586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:22.493380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b89d68-95c8-4311-bd8b-e1e2eccc0af4 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.504522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:22.530109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.530742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.531021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.531291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.600007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:22.601461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:22.702397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 523/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:22.704206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:22.808684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621052-28d7-41fd-a097-54051df7b7fe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.843641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.844550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.845033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.845540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.846872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:22.849270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64988a4-845a-476e-a957-2ae84df705a2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:22.856031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 529/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:22 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:22.857775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:22.870890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:22.960075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0a307-6479-4d8d-bcbc-55f14d3629d1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:22.993292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:22.993904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:22.994185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:22.994455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:22.995212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=1d75f5fa09d4402a8556e08bf8e18666, user.id=6d30d531408d49bd98102be9777e643c) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:22.995520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:22.996950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565bf82-9944-4c4d-881a-3c679f76fa64 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:23.008561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 524/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:22 2022] GET /identity/v3/role_assignments?user.id=6d30d531408d49bd98102be9777e643c&scope.project.id=1d75f5fa09d4402a8556e08bf8e18666&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:23.394577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320f19-8809-4d13-a76b-ecca55e5a7bf None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:23.395124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320f19-8809-4d13-a76b-ecca55e5a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:23.395124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320f19-8809-4d13-a76b-ecca55e5a7bf None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:23.396626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 530/1054] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:23.401553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:23.401841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:23.402115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:23.423611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:23.424784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:23.495166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 525/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:23.762813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82320f19-8809-4d13-a76b-ecca55e5a7bf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:23.826156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:23.868034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82320f19-8809-4d13-a76b-ecca55e5a7bf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:23.903904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:23.904532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:23.904808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:23.905076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:23.905651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:23.907106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-cdf52d64-42ca-4760-b609-5445571590e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:23.916250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:23.917389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 11:41:23.918247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:23.919964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 531/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Sep 12 11:41:23 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:23.923522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:23.927579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:24.029065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ca35-fd49-4b12-b6f3-f090f7c99522 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.043838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4d46-00b2-4fba-b562-f8f817a0099e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.044157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4d46-00b2-4fba-b562-f8f817a0099e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.044411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4d46-00b2-4fba-b562-f8f817a0099e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.045895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 532/1057] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:24.050468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.051303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.051584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.065512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.066172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.067013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.067309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.067923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] RBAC: Adding query filter params (name=image) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:24.068223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:24.069587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:24.070387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:24.071624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:24.074598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 526/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:24.080308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:24.149220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 533/1059] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:24.156653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:24.191946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc896a-6227-4fb6-b1b3-55e7f7029325 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.227049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.227654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.227936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.228205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.228824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] RBAC: Adding query filter params (type=image) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:24.229123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:24.230483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:24.240604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 527/1060] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:24.241877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:24.309624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:24.352809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb401d-38c1-4c74-a9e6-e5c19a79b16c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.386174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.386804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.387083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.387353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.388027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:24.389376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:24.418549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 534/1061] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:41:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:24.422864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:24.533744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c6ba-5d7d-4674-b457-bc0c87b8fd2c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.565974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:24.579513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.580154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.580427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.580692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.594045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:24.599432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:24.600452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:24.601696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 528/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:41:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:24.607895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:24.716524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775688c-6743-4ca7-ab61-31eb24c3f638 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.731071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:24.753734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.754346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.754809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.755088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.755753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:24.756052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:24.757496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:24.766455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 535/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:41:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:24.772883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:24.873891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fac136-2f40-4e87-92ce-e68d43f52e0a None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:24.907546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:24.908171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:24.908558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:24.908829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:24.919485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:24.920966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:24.936707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:24.938152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:24.939127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:24.939127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:24.939127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:24.939127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 529/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:24.945601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.046455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5710-c054-42b7-a713-498d23cee091 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.082144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.085867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.085867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.085867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.085867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:25.085867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.086312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6db08-b0e7-4b88-ad96-75c9a03e9f66 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.102728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 536/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:25.106738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.156190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.197457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614131c4-7181-4117-9737-cc32387d9074 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.197787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614131c4-7181-4117-9737-cc32387d9074 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.198022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614131c4-7181-4117-9737-cc32387d9074 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.199528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 537/1066] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:41:25.204230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.204508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.204779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.221004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1671f-2035-4770-9303-d0835a110427 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.223771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:25.225745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:25.256745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.258592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.259251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.259562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.259842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.267602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.269027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:25.281237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:25.282306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:25.283635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:25.287224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 530/1067] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:25.288041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.299609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 538/1068] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:25.391417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.395824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad64bc-e630-495f-87bf-96c08bd253e8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.429619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.430153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.430427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.430722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.431440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:25.431740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.433204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.450953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 531/1069] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:25.456626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.557017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e900e20-a671-4861-a710-e00c3d063c29 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.572755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.593028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.593800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.594176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.594571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.605806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.607333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:25.621954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:25.623277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:25.624178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:25.624178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:25.624178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:25.624178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 539/1070] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:25.629389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.651366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.684223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d375-77fe-4ddf-adee-9fa2d85dc1b3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.726147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.726827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.727114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.727389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.737600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:25.741577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604400a-b2e9-4541-8515-cf507d2c9ae0 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:25.743348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:25.744299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:25.745160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 532/1071] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:25.750306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.769542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.781916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.782540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.782833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.783105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.783742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:25.784046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.785924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.800160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 540/1072] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:25.806522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.861434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ade5c-5980-4999-a0f5-b150a4765d6e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.891967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:25.898035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.898679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.899056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.899234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.899903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:25.900208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.901583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.911168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 533/1073] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:25.916757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:25.920911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cd11a-018e-47d3-a959-b18c78688c79 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:25.957185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:25.957770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:25.958008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:25.958282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:25.965806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:25.967272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:25.979144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:25.980149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:25.985911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 541/1074] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:25.986449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:26.003139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:26.030137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771147d-d746-4606-818c-02700998cdf6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:26.064346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:26.064983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.065261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.065534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] PATH_INFO: `/v3/users/6d30d531408d49bd98102be9777e643c` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.066892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] RBAC: Authorizing `identity:get_user(user_id=6d30d531408d49bd98102be9777e643c)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:26.068344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:26.075179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 534/1075] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/users/6d30d531408d49bd98102be9777e643c => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:26.075784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:26.107557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33cfc-f682-4a6d-b9e3-d7a681a03ee6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:26.147675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:26.148295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.148574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.148851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.149556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:26.149857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:26.151309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:26.159162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:26.165275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 542/1076] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:26.170812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:26.189612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a347e-f578-46d2-876a-d8bfd14974e1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:26.226552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:26.227110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.227392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.227667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.228791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:26.230234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:26.231896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 535/1077] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:26 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:26.239315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:26.258024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:26.286194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021ebf3-5ac1-4152-803d-1dbd98a2828e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:26.325723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:26.326389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.326705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.326984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/44ed3e7aa7f3406398cf78e3f4f8c866` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.329296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=7e45de636b0e4f35aa279718cc129923, role_id=44ed3e7aa7f3406398cf78e3f4f8c866)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:26.330857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0157a-9ad8-4025-a9ed-018bfd2d7f76 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:26.345409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 543/1078] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:41:26 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/7e45de636b0e4f35aa279718cc129923/roles/44ed3e7aa7f3406398cf78e3f4f8c866 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:41:26.347551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c575-d536-45ab-a20f-85392be67ac8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:26.387606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:26.388635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.389096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.389608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666/users/6d30d531408d49bd98102be9777e643c/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.392054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] RBAC: Authorizing `identity:create_grant(project_id=1d75f5fa09d4402a8556e08bf8e18666, user_id=6d30d531408d49bd98102be9777e643c, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:26.393739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01909233-182c-4533-bfca-2befbba8c37e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:26.413550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 536/1079] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:41:26 2022] PUT /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666/users/6d30d531408d49bd98102be9777e643c/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:41:26.624786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660ac3-f0b0-4b87-b054-a0609d85739c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.625325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660ac3-f0b0-4b87-b054-a0609d85739c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.625325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660ac3-f0b0-4b87-b054-a0609d85739c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.626829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 544/1080] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:26.632724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:26.633011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:26.633282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:26.651868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:26.653038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:26.721118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 537/1081] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:26.986093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660ac3-f0b0-4b87-b054-a0609d85739c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:27.047587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:27.096526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660ac3-f0b0-4b87-b054-a0609d85739c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:27.135576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:27.136190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:27.136467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:27.136737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:27.206938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:27.208387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:27.266227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:27.314573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 545/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:27.320296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:27.328347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:27.422904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4ce7e-1167-43e1-a7c6-71a8c8112f1d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:27.456711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:27.457553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:27.457987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:27.458451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:27.469159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:27.470643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-9c745773-9f46-40a0-b636-23aeee2a426d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:27.485920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:27.486887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:27.488363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:27.488363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:27.488363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:27.490657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 538/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:27.493590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:27.594743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665584a-e656-4dff-8ab0-0140f98546e9 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:27.629392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:27.630006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:27.630264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:27.630664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:27.631257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:27.631523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:27.632824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000876-45d2-4257-9896-93e47de733b4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:27.650548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 546/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:28.558348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9eed-b659-4170-a3c1-1aacca9b76fe None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:28.558902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9eed-b659-4170-a3c1-1aacca9b76fe None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:28.558902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9eed-b659-4170-a3c1-1aacca9b76fe None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:28.560168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 539/1085] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:28.565497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:28.565786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:28.566706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:28.584978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:28.586149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:28.599117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8144-dc2b-46bc-92ff-c3a041f58b2d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:28.599404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8144-dc2b-46bc-92ff-c3a041f58b2d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:28.599587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:28.599683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8144-dc2b-46bc-92ff-c3a041f58b2d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:28.601093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 540/1086] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:28.606271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:28.606585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:28.606867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:28.625541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:28.626766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:28.654400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 547/1087] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:28.681271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:28.695142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 541/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 11:41:28.951056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:28.960330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.052435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19deda4c-2e08-4deb-b5dd-d20af0addd2f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.058810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9299-f1b2-4c44-8887-6688ec5a2244 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.089737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.090374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.090693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.090991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.092762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.093398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.093683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.093958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.101318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:29.104357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:29.106186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:29.107296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:29.108172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 548/1089] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Sep 12 11:41:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:29.109349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:29.110267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:29.115388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 542/1090] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:29.116344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.117343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.125902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.172084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.216706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d303a-7b2c-47ba-b366-e9cc539091a2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.222786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df1247-c4e2-43aa-b260-06ec8987dcb1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.250119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.250784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.251210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.251372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.252219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:29.252521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.254882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.257950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.258576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.258858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.259128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.259794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:29.260530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.261898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.263286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 549/1091] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:29.268752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.274869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 543/1092] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:29.275902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.341384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.373477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adc95-7dbf-492e-958d-9633e4f6f23e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.373793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252306cb-7fee-4131-afed-c94ce68b600f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.384054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.406932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.407556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.407837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.408110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] PATH_INFO: `/v3/users/6d30d531408d49bd98102be9777e643c` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.408918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.409234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] RBAC: Authorizing `identity:get_user(user_id=6d30d531408d49bd98102be9777e643c)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.410076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.410352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.410657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.411233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.412809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 544/1093] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/users/6d30d531408d49bd98102be9777e643c => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:29.418381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.423293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.424732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:29.439498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:29.440483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:29.442202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:29.442202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:29.442202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:29.442202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 550/1094] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:41:29.446702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.518810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9661-7275-44a7-980b-dd10056aaa89 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.546379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace1aba-adf3-4d63-9375-ccfd3a55ce7c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.551805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.552406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.552686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.552961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.569189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.582487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.583109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.583382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.583654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.584545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:29.584845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.586191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.598032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 551/1095] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:29.599609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.603675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.621636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.623109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.703581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e7993-79ae-46f3-b7f6-dfb9db1bb8a6 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.724763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 545/1096] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 11:41:29.731608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.745876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.746527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.746815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.747090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.754301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.755759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:41:29.767023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:29.768138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:29.769776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 552/1097] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:29.774178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.810732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.816657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:29.832768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb40f2-c9c1-4615-aece-b2f71775e1d3 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.865950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.866598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.866873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.867144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.868235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.869657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.871295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 546/1098] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:29.873610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf515-1c86-4557-a033-d253bad7a87f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:29.877339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.909952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:29.910606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:29.910889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:29.911827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:29.911941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:29.912176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:29.913583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:29.930984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 553/1099] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:29.936675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:29.975374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d698a7-20b2-4de8-a91f-3364746bc5a6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.009044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.009634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.009909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.010176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.010926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=1d75f5fa09d4402a8556e08bf8e18666, user.id=6d30d531408d49bd98102be9777e643c) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:30.011229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.012638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.018763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 547/1100] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/role_assignments?user.id=6d30d531408d49bd98102be9777e643c&scope.project.id=1d75f5fa09d4402a8556e08bf8e18666&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:30.033532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:30.037096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4604171-d3c6-4990-a911-500e0737d57e None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.076488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.077115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.077390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.077658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.151057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.152515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.253009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:30.256798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 554/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 11:41:30.262751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:30.275115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:30.360875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8f5dd-04ab-4d8b-bc49-2503c60bf124 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.393932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.394591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.394872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.395141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.405459 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.406923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:41:30.421226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:30.422343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:41:30.423285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:41:30.423285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:41:30.423285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:30.423285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 548/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:30.428683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:30.525784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652562e9-8abf-47b1-94ec-e7ce880f5f88 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.559256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.559858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.560137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.560407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.561040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:30.561342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.562722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.573960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 555/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:30.579404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:30.639402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:30.678959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283b01-0528-4303-a17c-3a521f50e2cf None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.712684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.713411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.713701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.713974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.720550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.722572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:41:30.733598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:30.734673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:30.735610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 549/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:30.740673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:30.799591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:30.841260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f692-1586-48b3-8dbe-a08fa9180c1d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:30.874771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:30.875373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:30.875653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:30.875933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:30.876619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:30.876927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:30.878949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202b4a1-cc88-4403-92d3-8b472c258f5d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:30.891059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 556/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:30.896774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:30.996259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0329c2-e49a-4a29-b6e0-bc18c6faadd2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:31.017686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:31.029971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:31.030595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:31.030877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:31.031147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:31.031873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] RBAC: Adding query filter params (role.id=44ed3e7aa7f3406398cf78e3f4f8c866, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=7e45de636b0e4f35aa279718cc129923) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:31.032181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:31.033634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809cc37-db75-4782-a918-1dece5a7ea1c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:31.039347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 550/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:30 2022] GET /identity/v3/role_assignments?user.id=7e45de636b0e4f35aa279718cc129923&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=44ed3e7aa7f3406398cf78e3f4f8c866 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:32.197752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c67d-76ab-451a-96aa-cb9f63ccb2cf None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:32.198258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c67d-76ab-451a-96aa-cb9f63ccb2cf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:32.198373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635c67d-76ab-451a-96aa-cb9f63ccb2cf None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:32.199656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 557/1107] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:32.204687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:32.205107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:32.205262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:32.225309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:32.226493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:32.290364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 551/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:32.558936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c67d-76ab-451a-96aa-cb9f63ccb2cf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:32.657761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635c67d-76ab-451a-96aa-cb9f63ccb2cf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:32.677082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:32.691496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:32.692112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:32.692391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:32.692663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:32.693775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:32.695228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288883f4-01c9-4fbe-8a62-f856be6c46d5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:32.696775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 558/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:32.702363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:32.797696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27791ad1-276c-4126-b2f3-0ce5a70b1869 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:32.813829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:32.831873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:32.832471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:32.832752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:32.833028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:32.833680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:32.835399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98708675-4dff-4ed1-8a1b-358590ed6819 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:32.855869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 552/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:41:32 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:41:35.055089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16af972-5200-44fc-8cdb-f995df967047 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:35.055623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16af972-5200-44fc-8cdb-f995df967047 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:35.055623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16af972-5200-44fc-8cdb-f995df967047 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:35.056966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 559/1111] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:35.062235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:35.062630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:35.062919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:35.081338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:35.082534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:35.147315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 553/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:35.412857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16af972-5200-44fc-8cdb-f995df967047 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:35.513062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16af972-5200-44fc-8cdb-f995df967047 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:35.546939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:35.547552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:35.547828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:35.548100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] PATH_INFO: `/v3/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:35.549471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] RBAC: Authorizing `identity:get_role(role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:35.551420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:35.552711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 560/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:35 2022] GET /identity/v3/roles/b42a9d0451fc419ab1547c88f1248317 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:35.558477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:35.652014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ddd0-88ad-4ce0-949f-0d17fbc6e4e3 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:35.653481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:35.686803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:35.687352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:35.687625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:35.687892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] PATH_INFO: `/v3/users/ac4567dcf86848b5ba51a4d5e33498a3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:35.702480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] RBAC: Authorizing `identity:get_user(user_id=ac4567dcf86848b5ba51a4d5e33498a3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:35.704126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:35.712014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 554/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:35 2022] GET /identity/v3/users/ac4567dcf86848b5ba51a4d5e33498a3 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:35.713237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:35.773372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:35.825518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d86cd-46cc-489d-97ad-1679d5b6c865 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:35.858915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:35.859513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:35.859789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:35.860063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:35.924743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:35.926323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:35.978619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:36.024374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 561/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 11:41:36.030201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:36.136360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272926d1-76b5-4c48-9f9a-a2ae4ba0f82f None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:36.169773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:36.170399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:36.170702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:36.170978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:36.172087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:36.173518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07444956-b77f-46e5-93b7-609123cbd106 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:36.175757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 555/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:36 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:36.181934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:36.280019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a22a1f-7f99-4b12-b92a-8873095d5687 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:36.312994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:36.313584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:36.313860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:36.314129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:36.314872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=1d75f5fa09d4402a8556e08bf8e18666, user.id=ac4567dcf86848b5ba51a4d5e33498a3) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:36.315175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:36.316696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445b01f-7bd9-417e-8268-fdc592973e69 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:36.321866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 562/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:36 2022] GET /identity/v3/role_assignments?user.id=ac4567dcf86848b5ba51a4d5e33498a3&scope.project.id=1d75f5fa09d4402a8556e08bf8e18666&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:38.533679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b29f8-6f39-4711-ba4a-2e9e20438893 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:38.534988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b29f8-6f39-4711-ba4a-2e9e20438893 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:38.534988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b29f8-6f39-4711-ba4a-2e9e20438893 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:38.536202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 556/1118] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:38.541387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:38.541769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:38.541934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:38.560235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:38.561403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:38.574864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:38.626409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 563/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:38.889031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b29f8-6f39-4711-ba4a-2e9e20438893 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:38.917841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:38.989316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b29f8-6f39-4711-ba4a-2e9e20438893 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:39.022390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:39.023032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:39.023310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:39.023580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] PATH_INFO: `/v3/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:39.024952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] RBAC: Authorizing `identity:get_role(role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:39.026315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:39.027640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 557/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:38 2022] GET /identity/v3/roles/b42a9d0451fc419ab1547c88f1248317 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:39.033288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:39.129445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bed21-f620-4b1e-9302-033f9d45e59f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:39.162152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:39.162736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:39.163016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:39.163285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] PATH_INFO: `/v3/users/ac4567dcf86848b5ba51a4d5e33498a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:39.164385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac4567dcf86848b5ba51a4d5e33498a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:39.165797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:39.167387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 564/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:39 2022] GET /identity/v3/users/ac4567dcf86848b5ba51a4d5e33498a3 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:39.173110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:39.263954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:39.270550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5973019-e5c8-4861-9aac-5a8eb64028ee None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:39.302918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:39.303530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:39.303808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:39.304077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:39.305170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:39.306619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def21561-b96f-42d9-a57d-0e607827f29b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:39.308221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 558/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:39 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:39.314074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:39.333798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:39.411198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cebed4-f03c-4d38-943a-0174a4f5e9d3 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:39.446004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:39.446596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:39.446874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:39.447143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666/users/ac4567dcf86848b5ba51a4d5e33498a3/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:39.449240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d75f5fa09d4402a8556e08bf8e18666, user_id=ac4567dcf86848b5ba51a4d5e33498a3, role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:39.450679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622762b0-2b03-43a1-8f05-f52bcff78949 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:39.463836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 565/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:41:39 2022] PUT /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666/users/ac4567dcf86848b5ba51a4d5e33498a3/roles/b42a9d0451fc419ab1547c88f1248317 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:41:41.572815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460240-2839-48cb-8325-bfb79d7dacb9 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:41.573326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460240-2839-48cb-8325-bfb79d7dacb9 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:41.573326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460240-2839-48cb-8325-bfb79d7dacb9 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:41.574635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 559/1124] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:41.579895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:41.580272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:41.580452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:41.598958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:41.600128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:41.664543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 566/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:41.931457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460240-2839-48cb-8325-bfb79d7dacb9 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:42.029770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460240-2839-48cb-8325-bfb79d7dacb9 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:42.062132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:42.062770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:42.063049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:42.063320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] PATH_INFO: `/v3/roles/b42a9d0451fc419ab1547c88f1248317` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:42.064688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] RBAC: Authorizing `identity:get_role(role_id=b42a9d0451fc419ab1547c88f1248317)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:42.066130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:42.067489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 560/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:41 2022] GET /identity/v3/roles/b42a9d0451fc419ab1547c88f1248317 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:42.073083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:42.084061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:42.172553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efc7d5-8692-4f55-9ce2-9a979e9f6735 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:42.207319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:42.207885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:42.208166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:42.208690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] PATH_INFO: `/v3/users/ac4567dcf86848b5ba51a4d5e33498a3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:42.209833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] RBAC: Authorizing `identity:get_user(user_id=ac4567dcf86848b5ba51a4d5e33498a3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:42.211323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:42.212906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 567/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:42 2022] GET /identity/v3/users/ac4567dcf86848b5ba51a4d5e33498a3 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:42.218800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:42.249456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:42.315914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6022a9b-e6f1-48f7-8632-44bb0e8f15e7 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:42.349081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:42.349665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:42.349946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:42.350218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:42.414114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:42.415577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:42.443623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:42.509148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 561/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:42.514942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:42.589208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:42.612677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8989a-9955-4750-9cde-d342b2bfe387 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:42.646568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:42.647197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:42.647479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:42.647752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] PATH_INFO: `/v3/projects/1d75f5fa09d4402a8556e08bf8e18666` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:42.648866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] RBAC: Authorizing `identity:get_project(project_id=1d75f5fa09d4402a8556e08bf8e18666)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:42.650281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217ff39-e776-4326-ac2b-b220fa09702e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:42.651943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 568/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:42 2022] GET /identity/v3/projects/1d75f5fa09d4402a8556e08bf8e18666 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:42.657926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:42.752151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d17f3c-8288-477f-a1a2-f27911795c29 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:42.780109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:42.785972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:42.786597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:42.786879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:42.787154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:42.787881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] RBAC: Adding query filter params (role.id=b42a9d0451fc419ab1547c88f1248317, scope.project.id=1d75f5fa09d4402a8556e08bf8e18666, user.id=ac4567dcf86848b5ba51a4d5e33498a3) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:42.788193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:42.789609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b98813-9302-4422-85b8-d5e507871d73 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:42.795292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 562/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:42 2022] GET /identity/v3/role_assignments?user.id=ac4567dcf86848b5ba51a4d5e33498a3&scope.project.id=1d75f5fa09d4402a8556e08bf8e18666&role.id=b42a9d0451fc419ab1547c88f1248317 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:44.919628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e8f6-947e-4096-9ca8-2cc3b2a3e6dc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:44.920164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e8f6-947e-4096-9ca8-2cc3b2a3e6dc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:44.920164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e8f6-947e-4096-9ca8-2cc3b2a3e6dc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:44.921519 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 569/1131] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:44.926647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:44.927064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:44.927188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:44.945663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:44.946866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:45.010469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 563/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:45.276132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e8f6-947e-4096-9ca8-2cc3b2a3e6dc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:45.374043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e8f6-947e-4096-9ca8-2cc3b2a3e6dc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:45.407485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:45.408103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:45.408418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:45.408692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:45.409808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:45.411250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ec57d-8753-4489-9cbe-5015c5fc1fe9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:45.412803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 570/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:45.418603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:45.513130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e57bd-c646-4201-8456-76f43e4207b8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:45.537744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:45.546426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:45.547054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:45.547329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:45.547597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:45.548240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:45.550174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da135b-601c-4d79-83a9-dc8c82274a32 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:45.576480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 564/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:41:45 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:41:47.779159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e48-4942-460b-bcca-63037b375364 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:47.779684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e48-4942-460b-bcca-63037b375364 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:47.779684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e48-4942-460b-bcca-63037b375364 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:47.781048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 571/1135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:47.786583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:47.787002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:47.787132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:47.806281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:47.807492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:47.870411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 565/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:48.140776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d8e48-4942-460b-bcca-63037b375364 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:48.155684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:48.241856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d8e48-4942-460b-bcca-63037b375364 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:48.274240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:48.274885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:48.275168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:48.275442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:48.276562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:48.277990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85d1f3-775a-48dd-8f0c-5fc6a1671456 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:48.279579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 572/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:41:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:48.285866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:48.381810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20a7e-9458-47af-b4f7-4884d2416034 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:48.390566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:48.415938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:48.416541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:48.416815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:48.417083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:48.417730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:48.419189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3ba21-40a3-4111-9aa6-c8ae381a17b3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:48.440055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 566/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:41:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:41:50.612045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79aea-9920-4e6a-a1ac-98e01889672f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:50.612613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79aea-9920-4e6a-a1ac-98e01889672f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:50.612613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79aea-9920-4e6a-a1ac-98e01889672f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:50.613943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 573/1139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:50.619170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:50.619570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:50.619720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:50.638019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:50.639223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:50.703586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 567/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:50.971150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79aea-9920-4e6a-a1ac-98e01889672f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.052067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:51.072253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79aea-9920-4e6a-a1ac-98e01889672f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:51.105326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:51.105952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:51.106233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:51.106505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:51.117242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:51.122048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:51.123251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:51.124523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 574/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:51.129941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.224015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:51.231050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b64fb3-c380-4db5-97a8-78033a1be62f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:51.264763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:51.265361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:51.265640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:51.265911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:51.266602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:51.266907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:51.268362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:51.276138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 568/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:51.282034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.379936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f8a86-63ce-4d25-910d-76abd9ae3aaa None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:51.412302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:51.412910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:51.413184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:51.413454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] PATH_INFO: `/v3/users/452d14769f584bcba588e32b08e6b0fc` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:51.423216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:51.426749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] RBAC: Authorizing `identity:get_user(user_id=452d14769f584bcba588e32b08e6b0fc)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:51.428175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:51.429744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 575/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:51 2022] GET /identity/v3/users/452d14769f584bcba588e32b08e6b0fc => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:51.435445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.539012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd8e7-cdde-4e97-b4e5-3db46afa22af None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:51.571682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:51.572278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:51.572555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:51.572826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:51.596230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:51.637600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:51.639075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:51.732913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 569/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:51.739054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.842757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b37dbd-7822-436d-b9d1-79a1b9a952cf None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:51.875228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:51.875844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:51.876122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:51.876395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] PATH_INFO: `/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:51.877993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] RBAC: Authorizing `identity:get_project(project_id=ab11f5a8ada24ade9a8829e71cb7d42e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:51.879437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202ddb4-4adc-41e1-9c93-5887cec294cc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:51.881058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 576/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:51 2022] GET /identity/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:51.887365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:51.941888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:51.984381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155e702-3a16-47f1-ad88-28cba3f740a1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:52.016660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:52.017837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:52.018128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:52.018487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:52.019245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=ab11f5a8ada24ade9a8829e71cb7d42e, user.id=452d14769f584bcba588e32b08e6b0fc) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:52.019554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:52.020973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368fe33-438e-4e48-811b-598617e6aa62 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:52.026319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 570/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:51 2022] GET /identity/v3/role_assignments?user.id=452d14769f584bcba588e32b08e6b0fc&scope.project.id=ab11f5a8ada24ade9a8829e71cb7d42e&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:54.131975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333b63-690c-40a1-9627-35bf6800c49e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:54.132411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333b63-690c-40a1-9627-35bf6800c49e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:54.132529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333b63-690c-40a1-9627-35bf6800c49e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:54.133886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 577/1147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:54.139147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:54.139568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:54.139698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:54.157959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:54.159166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:54.223936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 571/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:54.498927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9333b63-690c-40a1-9627-35bf6800c49e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:54.565084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:54.599494 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9333b63-690c-40a1-9627-35bf6800c49e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:54.632787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:54.633402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:54.633682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:54.633954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:54.644059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:54.648815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:54.649920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:41:54.651240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 578/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:54.655860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:54.703910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:54.751175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf24c6-08ce-481e-a426-3ea4875daaf5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:54.783570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:54.784166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:54.784445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:54.784713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:54.785371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:54.785671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:54.787060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:54.795326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 572/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:54.801754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:54.919601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aec2d-420d-44ce-9dea-8bf1354c81ca None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:54.945153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:54.953916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:54.954644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:54.955021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:54.955301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] PATH_INFO: `/v3/users/452d14769f584bcba588e32b08e6b0fc` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:54.956414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=452d14769f584bcba588e32b08e6b0fc)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:54.958086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:54.959680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 579/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:54 2022] GET /identity/v3/users/452d14769f584bcba588e32b08e6b0fc => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:54.965466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:55.063153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3630f-896c-4b3b-9a50-5e5b330f7161 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:55.075047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:55.096742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:55.097365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:55.097641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:55.097909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] PATH_INFO: `/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:55.099037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] RBAC: Authorizing `identity:get_project(project_id=ab11f5a8ada24ade9a8829e71cb7d42e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:55.100453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac53dd-c364-4095-91ac-8bb83c879752 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:55.102054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 573/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:54 2022] GET /identity/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:55.107923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:55.213844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c6396-803e-4045-84e4-fd24a45d1ae2 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:55.247168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:55.247804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:55.248080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:55.248349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] PATH_INFO: `/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e/users/452d14769f584bcba588e32b08e6b0fc/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:55.250215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab11f5a8ada24ade9a8829e71cb7d42e, user_id=452d14769f584bcba588e32b08e6b0fc, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:55.251709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa645d8-4fad-458d-9743-7ec81d055289 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:55.264660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 580/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:41:55 2022] PUT /identity/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e/users/452d14769f584bcba588e32b08e6b0fc/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:41:57.363526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd308645-41f2-4c26-a207-0e69bd632102 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:57.364084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd308645-41f2-4c26-a207-0e69bd632102 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:57.364084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd308645-41f2-4c26-a207-0e69bd632102 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:57.365448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 574/1154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:41:57.370496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:57.370930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:57.371072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:57.388745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:41:57.389907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:41:57.437615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:57.456114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 581/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:41:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:41:57.728631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd308645-41f2-4c26-a207-0e69bd632102 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:57.826266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd308645-41f2-4c26-a207-0e69bd632102 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:57.860158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:57.860836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:57.861116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:57.861389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:57.871585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:41:57.877006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:41:57.878283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:41:57.879434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 575/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:41:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:41:57.884069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:57.979741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adf080-31cb-47a7-80eb-bbc574edff72 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:58.012753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:58.013357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:58.013639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:58.013914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:58.014607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:58.014915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:58.016279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:58.024477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 582/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:41:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:58.030246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:58.071601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:58.128599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1b46-af16-4d77-ab9b-f0eb105571f6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:58.161192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:58.161814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:58.162099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:58.162375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] PATH_INFO: `/v3/users/452d14769f584bcba588e32b08e6b0fc` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:58.163526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] RBAC: Authorizing `identity:get_user(user_id=452d14769f584bcba588e32b08e6b0fc)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:58.164965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:58.166559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 576/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:41:58 2022] GET /identity/v3/users/452d14769f584bcba588e32b08e6b0fc => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:41:58.172271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:58.218255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:58.270989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3809a3-c81a-421c-ba2f-211e3e30c174 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:58.303246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:58.303838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:58.304117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:58.304390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:58.369113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:58.370563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:58.414323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:58.466163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 583/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:41:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:41:58.472747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:58.568990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc72de7-c1b6-4c36-9505-31ac1dd2cf6b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:58.577325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:41:58.602614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:58.603229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:58.603511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:58.603784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] PATH_INFO: `/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:58.604892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] RBAC: Authorizing `identity:get_project(project_id=ab11f5a8ada24ade9a8829e71cb7d42e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:58.606321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5620f1-258c-4d1c-b66b-8d649bfb9715 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:58.607967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 577/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:41:58 2022] GET /identity/v3/projects/ab11f5a8ada24ade9a8829e71cb7d42e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:41:58.614100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:41:58.711416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899760-e94d-4e6a-ac6d-c581b289e781 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:41:58.743782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:41:58.744366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:41:58.744639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:41:58.744904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:41:58.745618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=ab11f5a8ada24ade9a8829e71cb7d42e, user.id=452d14769f584bcba588e32b08e6b0fc) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:41:58.745918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:41:58.747334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec7021-969a-40f9-87dc-7fa9d5e8ffe0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:41:58.753483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 584/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:41:58 2022] GET /identity/v3/role_assignments?user.id=452d14769f584bcba588e32b08e6b0fc&scope.project.id=ab11f5a8ada24ade9a8829e71cb7d42e&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:00.887330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58633a9-98b5-4f5f-85cf-518580413c70 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:00.887865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58633a9-98b5-4f5f-85cf-518580413c70 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:00.887865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58633a9-98b5-4f5f-85cf-518580413c70 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:00.889200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 578/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:00.895217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:00.895603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:00.895756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:00.913753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:00.914960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:00.917629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:00.979673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 585/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:01.244878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58633a9-98b5-4f5f-85cf-518580413c70 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:01.345424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58633a9-98b5-4f5f-85cf-518580413c70 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:01.378205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:01.378824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:01.379107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:01.379382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:01.422348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:01.445677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:01.447167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:01.544762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 579/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:42:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:42:01.551226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:01.645534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5726b91-2081-407c-be0c-dbe3f1ac1271 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:01.677776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:01.678386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:01.678691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:01.678960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:01.685350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:01.686792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:01.689186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6822d6cf-151d-4976-89b4-2c6888aa9547 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:42:01.699232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:42:01.700589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:42:01.701785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 586/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:42:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:42:01.707012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:01.780975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:01.803830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b474-d81b-47f2-951c-40daea89f8d9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:01.839025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:01.840058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:01.840559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:01.841064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:01.842236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:01.842699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:01.844400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6107262-6a20-41f5-813a-b91c28407dc3 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:01.852483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 580/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Sep 12 11:42:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:04.005133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e6e3e-4e73-4e84-972d-4bdc869ac0aa None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:04.005698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e6e3e-4e73-4e84-972d-4bdc869ac0aa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:04.005698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e6e3e-4e73-4e84-972d-4bdc869ac0aa None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:04.007045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 587/1167] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:04.012226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fe0b-35ed-4593-90e7-502ebfb4c350 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:04.012623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fe0b-35ed-4593-90e7-502ebfb4c350 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:04.012801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985fe0b-35ed-4593-90e7-502ebfb4c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:04.031368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1985fe0b-35ed-4593-90e7-502ebfb4c350 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:04.032536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1985fe0b-35ed-4593-90e7-502ebfb4c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:04.096310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 581/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:04.363434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e6e3e-4e73-4e84-972d-4bdc869ac0aa None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:04.461700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e6e3e-4e73-4e84-972d-4bdc869ac0aa None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:04.493557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:04.494143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:04.494421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:04.494716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:04.495372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:04.496711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27418e7-f012-4329-af1b-afcb589b3af9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:04.511762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 588/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Sep 12 11:42:04 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:42:06.711243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0436-bb1b-469b-8803-c33d3f7772d7 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:06.711789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0436-bb1b-469b-8803-c33d3f7772d7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:06.711789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd0436-bb1b-469b-8803-c33d3f7772d7 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:06.713141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 582/1170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:06.718541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:06.718977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:06.719114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:06.737593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:06.738802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:06.792295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:06.804436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 589/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:07.073077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd0436-bb1b-469b-8803-c33d3f7772d7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:07.142119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:07.173056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd0436-bb1b-469b-8803-c33d3f7772d7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:07.205937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:07.206577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:07.206863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:07.207393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] PATH_INFO: `/v3/domains/0477dc05a4fc4fd8becdc23d0a0eaff0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:07.217439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=0477dc05a4fc4fd8becdc23d0a0eaff0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:07.218908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4b0ca-7fab-485f-a613-331c592d8cdd None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:07.221073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 583/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Sep 12 11:42:07 2022] GET /identity/v3/domains/0477dc05a4fc4fd8becdc23d0a0eaff0 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:07.226990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:07.321396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f16b2-a090-4b6d-af0e-29c32b796b57 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:07.354811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:07.355398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:07.355672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:07.355939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:07.356585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:07.357988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85aba9-5364-4a5d-a52a-fa0ad9adde44 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:07.383764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 590/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:42:07 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:42:09.517458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640557-defa-4d11-87c6-4898be21fc2b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:09.518088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640557-defa-4d11-87c6-4898be21fc2b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:09.518088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640557-defa-4d11-87c6-4898be21fc2b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:09.519355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 584/1174] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:09.524468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:09.524850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:09.525010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:09.543452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:09.544620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:09.578087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:09.609772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 591/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:09.875274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640557-defa-4d11-87c6-4898be21fc2b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:09.972674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640557-defa-4d11-87c6-4898be21fc2b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:10.000600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:10.007065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:10.007681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:10.007957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:10.008223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] PATH_INFO: `/v3/domains/0477dc05a4fc4fd8becdc23d0a0eaff0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:10.009318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=0477dc05a4fc4fd8becdc23d0a0eaff0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:10.010765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a6ce3-6227-47c5-af54-0b69d5fe0b3e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:10.012308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 585/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Sep 12 11:42:09 2022] GET /identity/v3/domains/0477dc05a4fc4fd8becdc23d0a0eaff0 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:10.018861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:10.115320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051cc3-8a66-47bf-9c77-82911ec2f98d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:10.148821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:10.149359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:10.149640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:10.149909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:10.150577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:10.152007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0b14-2dac-42ba-9521-09bf50aaefe9 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:10.172092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 592/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:42:10 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:42:12.331815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650fe47-95fb-4619-9f31-71727f64b18c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:12.332582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650fe47-95fb-4619-9f31-71727f64b18c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:12.332709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650fe47-95fb-4619-9f31-71727f64b18c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:12.334013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 586/1178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:12.339122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:12.339498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:12.339676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:12.358143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:12.359357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:12.381931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:12.425288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 593/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:12.691620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650fe47-95fb-4619-9f31-71727f64b18c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:12.791591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650fe47-95fb-4619-9f31-71727f64b18c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:12.823397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:12.824007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:12.824284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:12.824551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:12.835656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:42:12.840378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:42:12.841435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:42:12.842596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 587/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:42:12.848090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:12.946566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d09e3-ff65-457b-8b88-85f85e2ae299 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:12.979824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:12.980420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:12.980699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:12.980970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:12.981627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:12.981928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:12.983312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:12.991292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 594/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:12.997210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:13.055732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:13.096798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048de7-f6df-41ff-a408-3e40630ee848 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:13.129139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:13.129759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:13.130032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:13.130298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] PATH_INFO: `/v3/users/27e10263ec504fa282f7a670a4296626` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:13.143546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] RBAC: Authorizing `identity:get_user(user_id=27e10263ec504fa282f7a670a4296626)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:13.144970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:13.146546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 588/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:42:12 2022] GET /identity/v3/users/27e10263ec504fa282f7a670a4296626 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:13.152255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:13.174275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:13.249820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d843ae-16c8-417d-aca6-fc169618cea8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:13.282130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:13.282803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:13.283088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:13.283361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:13.347568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:13.348997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:13.369916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:13.444851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 595/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:42:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:42:13.450751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:13.545737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29f26-58ac-423c-add6-282bc6e7fdfb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:13.561819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:13.579594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:13.580202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:13.580479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:13.580747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] PATH_INFO: `/v3/projects/08c3baccafee44d488b5486505f90fa6` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:13.582123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] RBAC: Authorizing `identity:get_project(project_id=08c3baccafee44d488b5486505f90fa6)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:13.583573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9884735-6c83-47c9-bb38-2027d45ef10b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:13.585168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 589/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:42:13 2022] GET /identity/v3/projects/08c3baccafee44d488b5486505f90fa6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:13.591240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:13.687066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150f5a5-a382-43f3-8874-e2b6cdb34370 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:13.706127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:13.720839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:13.721423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:13.721697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:13.721961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:13.722706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=08c3baccafee44d488b5486505f90fa6, user.id=27e10263ec504fa282f7a670a4296626) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:13.723009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:13.724397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82989c7e-5d6b-47f4-a58d-5cc39720b24e None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:13.729588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 596/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:42:13 2022] GET /identity/v3/role_assignments?user.id=27e10263ec504fa282f7a670a4296626&scope.project.id=08c3baccafee44d488b5486505f90fa6&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:15.850675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b36f-5050-4011-a52b-521d2b98c91d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:15.851229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b36f-5050-4011-a52b-521d2b98c91d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:15.851229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b36f-5050-4011-a52b-521d2b98c91d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:15.852551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 590/1186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:15.857710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:15.858136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:15.858256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:15.876678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:15.877851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:15.943680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 597/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:16.216389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b36f-5050-4011-a52b-521d2b98c91d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:16.316514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b36f-5050-4011-a52b-521d2b98c91d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:16.350061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:16.350721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:16.351003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:16.351276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:16.361090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:42:16.365893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:42:16.367040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:42:16.367899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 591/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:42:16.372943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:16.469050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acd672-66ec-47c2-bf8d-fe66dc8160d9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:16.483022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:16.502973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:16.503580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:16.503859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:16.504130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:16.504795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:16.505097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:16.506452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:16.514392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 598/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:16.520918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:16.566225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:16.620231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288801e-07b4-4640-b07c-192377fe0c03 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:16.653428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:16.654050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:16.654327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:16.654628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] PATH_INFO: `/v3/users/27e10263ec504fa282f7a670a4296626` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:16.655745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] RBAC: Authorizing `identity:get_user(user_id=27e10263ec504fa282f7a670a4296626)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:16.657384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:16.658982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 592/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:42:16 2022] GET /identity/v3/users/27e10263ec504fa282f7a670a4296626 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:16.664862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:16.761927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88d12-8b52-4117-8ffc-d77ff7fdbc0b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:16.794205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:16.794848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:16.795129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:16.795402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] PATH_INFO: `/v3/projects/08c3baccafee44d488b5486505f90fa6` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:16.796512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] RBAC: Authorizing `identity:get_project(project_id=08c3baccafee44d488b5486505f90fa6)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:16.797926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37faa-6451-4cdd-8e5d-0f3dd3772c34 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:16.799725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 599/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:42:16 2022] GET /identity/v3/projects/08c3baccafee44d488b5486505f90fa6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:16.805541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:16.902206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702b29-be23-4a03-a80e-800406f165cb None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:16.914767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:16.935427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:16.936070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:16.936346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:16.936619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] PATH_INFO: `/v3/projects/08c3baccafee44d488b5486505f90fa6/users/27e10263ec504fa282f7a670a4296626/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:16.939249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=08c3baccafee44d488b5486505f90fa6, user_id=27e10263ec504fa282f7a670a4296626, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:16.940682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90b70e-05f1-498d-b14f-b0ebfe2c2d30 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:16.953191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 593/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:42:16 2022] PUT /identity/v3/projects/08c3baccafee44d488b5486505f90fa6/users/27e10263ec504fa282f7a670a4296626/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:42:19.051045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a87c-9f4a-4aab-bbe7-cface33ac9bd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.051636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a87c-9f4a-4aab-bbe7-cface33ac9bd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.051636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29a87c-9f4a-4aab-bbe7-cface33ac9bd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:19.052922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 600/1193] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:42:19.058123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.058559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.058702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:19.077605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:42:19.079371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:42:19.143922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 594/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:42:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:42:19.408582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29a87c-9f4a-4aab-bbe7-cface33ac9bd None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:19.454073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:19.507773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29a87c-9f4a-4aab-bbe7-cface33ac9bd None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:19.540657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395365-322b-4546-866f-715f9df29331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:19.541269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395365-322b-4546-866f-715f9df29331 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.541544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395365-322b-4546-866f-715f9df29331 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.541813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395365-322b-4546-866f-715f9df29331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:19.553008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395365-322b-4546-866f-715f9df29331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-33395365-322b-4546-866f-715f9df29331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:42:19.557710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:42:19.558867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:42:19.560155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 601/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:42:19.564992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:19.661612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4235f5f-e5d9-4666-b3ee-d5fd8dff5f6c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:19.663056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:19.697026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:19.697624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.697902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.698169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:19.698862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:19.699163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:19.700534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:19.709487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 595/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:19.715422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395365-322b-4546-866f-715f9df29331 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:19.811044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395365-322b-4546-866f-715f9df29331 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:19.813941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:19.844547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:19.845157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.845433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.845701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] PATH_INFO: `/v3/users/27e10263ec504fa282f7a670a4296626` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:19.846835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] RBAC: Authorizing `identity:get_user(user_id=27e10263ec504fa282f7a670a4296626)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:19.848233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:19.849770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 602/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Sep 12 11:42:19 2022] GET /identity/v3/users/27e10263ec504fa282f7a670a4296626 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:19.855448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:19.957538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8870c7-ae8c-4ce1-b2cb-dcdc98ec1f4b None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:19.990060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:19.990689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:19.990968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:19.991240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:20.020877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:20.056439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:20.057877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:20.153331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 596/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:42:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:42:20.159344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:20.259629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af45a6f-5b07-4772-b1bd-5c4f34d48d74 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:20.291499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:20.292116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:20.292391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:20.292662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] PATH_INFO: `/v3/projects/08c3baccafee44d488b5486505f90fa6` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:20.293764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] RBAC: Authorizing `identity:get_project(project_id=08c3baccafee44d488b5486505f90fa6)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:20.295208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07836a64-9a7d-432a-8c02-0f0b36c3fd10 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:20.296815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 603/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Sep 12 11:42:20 2022] GET /identity/v3/projects/08c3baccafee44d488b5486505f90fa6 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:20.303630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:42:20.363196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:20.398901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c55e-5149-45b9-87fa-bcc9e3c857e8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:42:20.431908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:42:20.432488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:42:20.432765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:42:20.433030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:42:20.433743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=08c3baccafee44d488b5486505f90fa6, user.id=27e10263ec504fa282f7a670a4296626) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:42:20.434044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:42:20.435479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b90e-295e-4bf9-826f-0020a91251c9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:42:20.441436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 597/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:42:20 2022] GET /identity/v3/role_assignments?user.id=27e10263ec504fa282f7a670a4296626&scope.project.id=08c3baccafee44d488b5486505f90fa6&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:42:30.287460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=80 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:42:30.438909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=36 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:16.363759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9257f4-8b42-4f99-a02f-fae9ee222eb9 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:16.363759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9257f4-8b42-4f99-a02f-fae9ee222eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:16.363759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9257f4-8b42-4f99-a02f-fae9ee222eb9 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:16.363759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 604/1201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:16.369152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:16.369903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:16.369903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:16.389962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:16.391291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:16.486600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 598/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:16.763385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9257f4-8b42-4f99-a02f-fae9ee222eb9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:16.892079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9257f4-8b42-4f99-a02f-fae9ee222eb9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:16.938221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:16.939025 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:16.939291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:16.939623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:16.958965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:16.960768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:16.981429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:16.982755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:16.983868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:16.983868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:16.983868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:16.986204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 605/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:16.992455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:17.127718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18fc41-8c69-4ecf-84c5-8e32a2877d0b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:17.169999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:17.181754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:17.182454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:17.182803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:17.183099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:17.183808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:17.184091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:17.185474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1d31-ddb4-465d-83a5-0d20517682eb None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:17.203896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 599/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:17.205094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:17.283314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:17.340869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174e34d-d3f9-44a6-a18d-38556842f6fe None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:17.384361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:17.385779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:17.386110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:17.387019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:17.387766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:17.389356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62230ae3-0dbf-437a-aee6-5f3084ce076b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:17.414609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 606/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Sep 12 11:43:17 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:43:19.906725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86d56-32b2-4dad-86b1-47977fd5c33c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:19.907310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86d56-32b2-4dad-86b1-47977fd5c33c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:19.907310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b86d56-32b2-4dad-86b1-47977fd5c33c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:19.909057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 600/1206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:19.914434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:19.914918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:19.915097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:19.936116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:19.937305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:20.017233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 607/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:20.315896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b86d56-32b2-4dad-86b1-47977fd5c33c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:20.442802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b86d56-32b2-4dad-86b1-47977fd5c33c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:20.483474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:20.484112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:20.484390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:20.484806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:20.498060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:20.503915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:20.505269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:20.506129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 601/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:43:20.511597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:20.559089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:20.626845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5c911-d9f0-4e1e-8885-82d9534015d8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:20.665549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:20.666202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:20.666479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:20.666815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:20.668078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:20.668348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:20.669713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:20.685508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 608/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:20.687539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:20.756584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:20.824628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fca05-3d53-4aee-a33e-204e71ff7de8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:20.861904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:20.862557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:20.862860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:20.863109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:20.875408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:20.877015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:20.892931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:20.894160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:20.895264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:20.895264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:20.895264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:20.895264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 602/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:20.900468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:21.010911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:21.014301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfe953-2e58-400c-932c-315eff9b4c2d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:21.053526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:21.054180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:21.054539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:21.054920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:21.055684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:21.056142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:21.058532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:21.073486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 609/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:21.079502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:21.199444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a9d-4f41-4701-b67a-f094f0d07a23 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:21.201574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:21.243106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:21.243847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:21.244240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:21.244550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:21.256505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:21.257962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:21.270327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:21.271410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:21.272443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:21.277728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 603/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:43:21.277848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:21.380532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1421b5-dff3-451b-8e01-74bbd478d74a None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:21.416637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:21.417361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:21.417641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:21.417912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:21.418703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:21.419011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:21.420424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:21.443723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 610/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:21.445040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:21.569966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f179873-8dd4-451d-9532-c32df24a1ff8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:21.617568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:21.618195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:21.618491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:21.618919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:21.659498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:21.710416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:21.711948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:21.855461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 604/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:43:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:43:21.857046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:21.888097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:21.984533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8a8a-2609-4497-bc79-2a4df2e1e11d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:22.021379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:22.022267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:22.022267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:22.022539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:22.033146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:22.034612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:22.050098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:22.051279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:22.052202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:22.052202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:22.052202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:22.052202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 611/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:22.057098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:22.125353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:22.170803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ab79-3a58-4f03-b761-bebbd5bccae2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:22.213015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:22.213674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:22.214504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:22.214969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:22.215964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:22.216308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:22.217815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:22.230928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 605/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:22.236648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:22.311725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:22.347321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f40cb-1e27-4e0c-a0eb-0ed61b955148 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:22.384918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:22.385503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:22.385783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:22.386748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:22.394999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:22.396500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:22.409283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:22.410400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:22.411574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:22.417031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 612/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:22.418823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:22.532490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec8ded-5110-4230-9a82-179d5d2b16f6 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:22.557279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:22.571302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:22.572097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:22.572379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:22.572653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:22.573389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:22.573683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:22.575289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750ca3b-c4d2-464f-b185-471cff322983 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:22.595034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 606/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:22.596358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:22.712575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19be5b-9e76-4e48-a2d8-f394a7a0ba21 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:22.747062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:22.754100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:22.754799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:22.755082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:22.755398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:22.756197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=27b96a89a0d140aeaeba8078a21b4cbf) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:22.756537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:22.758118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50938c-571e-490c-8d90-478d7cab7c95 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:22.771907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 613/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:43:22 2022] GET /identity/v3/role_assignments?user.id=27b96a89a0d140aeaeba8078a21b4cbf&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:24.916748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980a05-4dca-4c3a-b6ee-a66c4751b7ed None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:24.917320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980a05-4dca-4c3a-b6ee-a66c4751b7ed None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:24.917460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9980a05-4dca-4c3a-b6ee-a66c4751b7ed None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:24.918714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 607/1220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:24.924249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:24.924728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:24.924824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:24.946867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:24.948790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:25.028000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 614/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:25.326582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9980a05-4dca-4c3a-b6ee-a66c4751b7ed None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:25.437282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9980a05-4dca-4c3a-b6ee-a66c4751b7ed None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:25.475078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:25.475904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:25.476210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:25.476597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:25.490041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:25.499900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:25.500874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:25.501738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:25.502387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 608/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:43:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:25.510686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:25.623670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44a85b-e40d-4677-b4fc-df510d1f5a76 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:25.646821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:25.660899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:25.661625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:25.661879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:25.662190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:25.663003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:25.663345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:25.664818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:25.674118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 615/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:43:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:25.680195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:25.736184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:25.798137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce4b6c-e795-4e66-8d26-8f5100b5a231 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:25.833010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:25.833730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:25.834082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:25.834424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:25.845776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:25.847244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:25.863601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:25.864969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:25.866222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:25.866222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:25.866222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:25.866222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 609/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:25.871399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:25.992673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824f80-f0e9-4c06-9a47-27cf5f2fdaa7 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.028034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.029585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.029585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.029585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.029833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:26.030062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.031565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.044514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 616/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:26.050330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:26.155592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:26.162951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52c6ec-eef1-43e7-a7d8-c29d3671fdd2 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.199683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.200348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.200632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.200914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.209757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.211271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:26.223442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:26.224591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:26.225569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 610/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:26.230565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:26.282483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:26.340342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48293-abf5-4b5a-80c4-844a17f47d52 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.377093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.377756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.378036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.378309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.379077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:26.379381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.380797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.401635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 617/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:26.407609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:26.516160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bceaa-7622-4f3b-917f-e9e01953b226 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.559392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.560101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.560462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.560833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.574772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.576352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:26.595428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:26.596770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:26.597994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:26.597994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:26.597994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:26.597994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 611/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:43:26.603885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:26.724771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30503720-bab9-4cb9-a602-f8ad6b786753 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.727605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:26.764844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.765458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.765737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.766017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.766698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:26.767035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.768400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.780882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 618/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:26.786975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:26.798000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:26.897565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3531af3-9aa1-4606-88c1-42d7a34d1759 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:26.934932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:26.935566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:26.935849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:26.936120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:26.942984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:26.944577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:26.958219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:26.959294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:26.960678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 612/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:26.965782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:27.077008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623f69a-362c-4606-952b-a1f8eb378e8d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:27.116633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:27.117296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:27.117577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:27.117853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:27.118581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:27.118892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:27.120433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777baf18-3fed-4789-91bb-9f292fc17da5 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:27.133978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 619/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:27.140224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:27.216303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:27.254153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d079d14-3415-4f6a-92bf-845229932cb8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:27.289202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:27.289850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:27.290130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:27.290409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/27b96a89a0d140aeaeba8078a21b4cbf/roles/34a321aa912c4977934cf9848dbe8ff0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:27.306759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=27b96a89a0d140aeaeba8078a21b4cbf, role_id=34a321aa912c4977934cf9848dbe8ff0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:27.308514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aa131-34a1-4c3a-92c2-1d736074625e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:27.321470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 613/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:43:27 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/27b96a89a0d140aeaeba8078a21b4cbf/roles/34a321aa912c4977934cf9848dbe8ff0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:43:29.637908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a1a0-27e4-4bba-b32d-7976cd8b3a83 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:29.638704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a1a0-27e4-4bba-b32d-7976cd8b3a83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:29.638704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a1a0-27e4-4bba-b32d-7976cd8b3a83 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:29.640053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 620/1233] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:29.645468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:29.645911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:29.646016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:29.665385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:29.667126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:29.746759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 614/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:30.020240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718a1a0-27e4-4bba-b32d-7976cd8b3a83 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:30.050713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:30.136644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718a1a0-27e4-4bba-b32d-7976cd8b3a83 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:30.174229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:30.174895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:30.175254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:30.175490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:30.186808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:30.191924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:30.194880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:30.196062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 621/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:30.199443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:30.250208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:30.310455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4ea3-647a-457b-a64c-f6260a9e4be7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:30.346423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:30.347059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:30.347342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:30.347616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:30.348292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:30.348599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:30.349977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:30.358374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 615/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:30.364385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:30.459502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:30.479613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc127bbd-ad75-42da-86b4-d52ef544aa53 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:30.521261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:30.522085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:30.522468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:30.522812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:30.534994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:30.536731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:30.555463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:30.561872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:30.563068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:30.563068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:30.563068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:30.563068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 622/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:30.563356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:30.677575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50960-a14f-48f6-996e-e5475d0d7ec8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:30.687628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:30.717819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:30.718382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:30.718705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:30.718978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:30.719710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:30.720022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:30.721645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:30.736247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 616/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:30.742833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:30.857989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73050e-8d35-4054-a88b-f7bbf5cb62be None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:30.895187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:30.895926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:30.896205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:30.896504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:30.899472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:30.906033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:30.907784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:30.920000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:30.925957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:30.926956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 623/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:30.927606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:31.036999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a488e7-1d85-4ff1-b99a-56fccd628926 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:31.073637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:31.074244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:31.074546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:31.074816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:31.075704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:31.076039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:31.078190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:31.099607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 617/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:31.106356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:31.214711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a7468-3234-4df2-8c8e-b21bd085558b None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:31.253531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:31.254256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:31.254684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:31.255071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:31.331054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:31.338533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:31.340631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:31.453337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 624/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:43:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:43:31.459536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:31.490193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:31.576839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca3a89-fdfc-428e-bf19-4565cc7bfb07 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:31.620239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:31.621094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:31.621376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:31.621744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:31.633146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:31.634652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:31.649781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:31.656272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:31.657484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:31.657484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:31.657484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:31.657484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 618/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:31.657774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:31.762773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:31.770024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3845d7-fe06-4eeb-ab97-e8fab7fd3f76 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:31.809928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:31.810575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:31.810858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:31.811127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:31.811793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:31.812129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:31.813524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:31.826548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 625/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:31.831838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:31.920812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:31.945323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd503e-47ac-47ba-967a-eadd688fe59e None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:31.982490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:31.983161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:31.983473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:31.983790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:31.991263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:31.993219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:32.006692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:32.007739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:32.008841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 619/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 11:43:32.016262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:32.131700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaaf41-d930-4fa5-9306-2f08783a06e4 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:32.173255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:32.173902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:32.174185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:32.174460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:32.175186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:32.175493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:32.176925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3195f1-b3c7-418f-91b8-6cfd20e36085 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:32.192154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 626/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:32.198643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:32.312037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f17dbc-2062-44f2-9693-813d8fb19d75 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:32.349365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:32.349976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:32.350255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:32.350572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:32.351427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] RBAC: Adding query filter params (role.id=34a321aa912c4977934cf9848dbe8ff0, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=27b96a89a0d140aeaeba8078a21b4cbf) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:32.351706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:32.353150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27cbd2-7bed-4b85-8ab4-2cda682c0302 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:32.359417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 620/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:43:32 2022] GET /identity/v3/role_assignments?user.id=27b96a89a0d140aeaeba8078a21b4cbf&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=34a321aa912c4977934cf9848dbe8ff0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:34.832508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26bb6b-a5c0-47e2-a1c0-aef69f531e0f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:34.833047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26bb6b-a5c0-47e2-a1c0-aef69f531e0f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:34.833047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26bb6b-a5c0-47e2-a1c0-aef69f531e0f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:34.834390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 627/1247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:34.840061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:34.840484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:34.840645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:34.847585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:34.862423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:34.863744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:34.947759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 621/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:35.233981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26bb6b-a5c0-47e2-a1c0-aef69f531e0f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:35.252975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:35.348718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26bb6b-a5c0-47e2-a1c0-aef69f531e0f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:35.387775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:35.388431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:35.388796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:35.389138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:35.401421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:35.407215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:35.408376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:35.409616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 628/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:35.414317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:35.521121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ddfbc-5248-45fc-819d-fde925755ae7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:35.554636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:35.562587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873103c3-7408-4065-9086-911a5af25164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:35.563284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873103c3-7408-4065-9086-911a5af25164 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:35.563635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873103c3-7408-4065-9086-911a5af25164 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:35.563981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873103c3-7408-4065-9086-911a5af25164 None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:35.564660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873103c3-7408-4065-9086-911a5af25164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:35.564956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873103c3-7408-4065-9086-911a5af25164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:35.566354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873103c3-7408-4065-9086-911a5af25164 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:35.575164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 622/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:35.581856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:35.669839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:35.695547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb729-083f-4539-96bd-03e6650f6d78 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:35.738764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:35.739404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:35.739679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:35.739946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:35.752625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:35.754324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:35.772038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:35.773334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:35.774462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:35.774462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:35.774462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:35.774462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 629/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:35.779120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873103c3-7408-4065-9086-911a5af25164 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:35.888018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873103c3-7408-4065-9086-911a5af25164 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:35.924933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:35.925463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:35.925742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:35.926014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:35.926709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:35.927012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:35.928413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:35.941387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 623/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:35.947863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:36.055921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504e15d-dd29-477b-a839-6a5722cc477c None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:36.084067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:36.093557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:36.094199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:36.094479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:36.094796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:36.103145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:36.104647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:36.116825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:36.117967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:36.120024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 630/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:36.124661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:36.183004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:36.229268 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc1d31-fe5a-4d54-a5f3-9ff7866eced9 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:36.267312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:36.267920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:36.268199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:36.268471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:36.269221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:36.269522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:36.270997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:36.289009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 624/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:36.295652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:36.405476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a857c5b-6300-46ea-ac4d-493c87df39ab None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:36.439304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:36.439830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:36.440103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:36.440371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:36.496107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:36.513032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:36.514504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:36.631099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 631/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:43:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 11:43:36.639630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:36.750191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad807ba-15ae-4781-8849-3226452c6468 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:36.765171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:36.785667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:36.786293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:36.786602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:36.786874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:36.796972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:36.798393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:36.812889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:36.814031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:36.815210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:36.815210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:36.815210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:36.815210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 625/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:36.819683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:36.902733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:36.917224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780741-21de-4ade-a6bf-70378139d4bc None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:36.951263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:36.951873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:36.952120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:36.952384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:36.953264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:36.953560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:36.954927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:36.965795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 632/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:36.971457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:37.068705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8656-08c4-4207-8b5e-bbcb1d95bc14 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:37.101578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:37.102200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:37.102477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:37.102774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:37.109431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:37.110921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:37.123512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:37.124573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:37.125680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 626/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:37.130329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:37.230430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b85bc-9110-4e79-89ab-7345e5b13de8 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:37.265274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:37.265875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:37.266155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:37.266427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:37.267146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:37.267451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:37.268870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490ea71-44b6-4280-a060-6a5cef903f30 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:37.280643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 633/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:37.286749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:37.312504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:37.384548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2607-f184-486a-98bc-dab280a55977 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:37.417701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:37.418283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:37.418588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:37.418865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:37.419587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=27b96a89a0d140aeaeba8078a21b4cbf) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:37.419894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:37.421315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eccc6-11a8-4e61-acb8-776459e7c22c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:37.426558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 627/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:43:37 2022] GET /identity/v3/role_assignments?user.id=27b96a89a0d140aeaeba8078a21b4cbf&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:39.541222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce4c85-efe1-475e-98bc-7d40b59df561 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:39.541755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce4c85-efe1-475e-98bc-7d40b59df561 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:39.541755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce4c85-efe1-475e-98bc-7d40b59df561 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:39.543339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 634/1261] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:39.549636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:39.550088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:39.550207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:39.569134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:39.570320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:39.640797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 628/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:39.949556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce4c85-efe1-475e-98bc-7d40b59df561 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:39.960804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.053685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce4c85-efe1-475e-98bc-7d40b59df561 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.088133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.088932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.089213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.089487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.099945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:40.105611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:40.106802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:40.107760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 635/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:43:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:40.112834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:40.134176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.226052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ec1c9-c850-4706-8e95-15917bd0146d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.264103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.264712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.264991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.265262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.265924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:40.266224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:40.267626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:40.276140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 629/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:40.282182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:40.348996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.380517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0821-0513-4212-8d26-6f199eb266cd None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.413328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.413942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.414218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.414486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.424556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:40.425972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:40.440053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:40.441175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:40.442099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:40.442099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:40.442099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:40.442099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 636/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:40.447565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:40.526868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.546027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d0f3e-19eb-4919-ba10-fe0b5c1fb37c None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.579040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.579645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.579901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.580171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.580806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:40.581109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:40.582468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:40.594591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 630/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:40.600252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:40.698098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6664e3-6805-4fb0-adf0-c101df7c8428 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.719774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.736317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.736945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.737224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.737495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.745521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:40.746974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:40.757813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:40.758967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:40.760907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 637/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:40.765440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:40.862709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4f2e3-cd30-4225-93b2-6e2af147045d None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:40.895535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:40.896117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:40.896393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:40.896656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:40.897366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:40.897665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:40.899103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:40.907403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:40.916688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 631/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:40.922404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:41.019791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23d89c-6c20-455c-ab16-2936d193bc63 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:41.051590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:41.052195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:41.052470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:41.052739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:41.063401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:41.064824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:41.078618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:41.079557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:41.080563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:41.080563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:41.080563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:41.080563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 638/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:41.085261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:41.180590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2719e54-23a8-4203-8f8b-17668716e2c1 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:41.213174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:41.213759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:41.214034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:41.214301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:41.214964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:41.215253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:41.216604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:41.227199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 632/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:41.233531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:41.264657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:41.330796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92ee8f-cc01-4591-a22c-29b42d29c02f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:41.362982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:41.363588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:41.363865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:41.364140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:41.371174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:41.372528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:41.383287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:41.384867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:41.385942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:41.386543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 639/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:41.390710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:41.445964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:41.486750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eec0d-94ca-4d8a-80d3-2cf9442e6609 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:41.518276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:41.518900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:41.519181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:41.519454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:41.520149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:41.520452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:41.521873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce66c9-8fe9-4d26-995e-faf55030cb41 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:41.534075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 633/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:41.539855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:41.630108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:41.638971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baba37-5fdf-4d3e-a093-d94bf5769434 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:41.671982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:41.672624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:41.672904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:41.673176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] PATH_INFO: `/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/27b96a89a0d140aeaeba8078a21b4cbf/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:41.675072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] RBAC: Authorizing `identity:create_grant(project_id=77204d1f5d484097a21762f2a6bef64c, user_id=27b96a89a0d140aeaeba8078a21b4cbf, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:41.676493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de9f62-d8b1-4912-9ce4-dee47e21d81d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:41.688327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 640/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Sep 12 11:43:41 2022] PUT /identity/v3/projects/77204d1f5d484097a21762f2a6bef64c/users/27b96a89a0d140aeaeba8078a21b4cbf/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:43:43.846716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bcdff-b449-4c78-addc-964215a486af None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:43.847275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bcdff-b449-4c78-addc-964215a486af None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:43.847275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bcdff-b449-4c78-addc-964215a486af None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:43.848607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 634/1274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:43.853826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:43.854214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:43.854368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:43.872546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:43.873700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:43.937424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 641/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:44.201476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bcdff-b449-4c78-addc-964215a486af None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:44.301317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bcdff-b449-4c78-addc-964215a486af None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:44.318057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:44.335360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:44.335981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:44.336264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:44.336540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:44.347107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 11:43:44.351904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:44.353121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:44.354205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 635/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:44.358986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:44.436670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:44.456104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f72815-41a0-4793-b05a-ec4189cec1d6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:44.488671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:44.489267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:44.489546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:44.489821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:44.490483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:44.490817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:44.492176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:44.500542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 642/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:44.506408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:44.603365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd2c0-e07d-498d-92c5-50a10ba62e95 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:44.636556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:44.637164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:44.637443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:44.637715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:44.647350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:44.648805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:44.662894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:44.664170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:44.665424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:44.665424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:44.665424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:44.665424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 636/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:44.669629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:44.784058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24c787-bec2-4f71-b01c-b11c0c651f43 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:44.819071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:44.819667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:44.819943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:44.820213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:44.820843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:44.821143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:44.822663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:44.834108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 643/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:44.839883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:44.871298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:44.937775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08311d2d-08f3-40df-ac34-2af950755a67 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:44.969238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:44.969843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:44.970117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:44.970381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:44.977544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:44.979641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 11:43:44.990277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:44.991442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:44.992718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 637/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:44.997942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.001741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:45.094423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb5aa-d1b2-4b08-8f48-6aa313e6185f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:45.127574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:45.128170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:45.128446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:45.128718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:45.129433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:45.129732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:45.131181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:45.147522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 644/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Sep 12 11:43:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:45.153792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.241735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:45.251434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88437e66-29df-4e2a-bdf9-06ee0ed266e8 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:45.283805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:45.284396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:45.284675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:45.284943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:45.349705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:45.351120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:45.435442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:45.445513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 638/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Sep 12 11:43:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 11:43:45.451449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.512150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:45.549463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0dc9d-58f1-4663-9fc2-d70a8e564335 None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:45.582908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:45.583527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:45.583801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:45.584065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:45.594373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:45.595823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 11:43:45.609819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:45.610787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 11:43:45.611795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 11:43:45.611795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 11:43:45.611795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:45.611795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 645/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:45.616618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.715350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd7754-d6e8-44f7-abec-d202a4c3ee40 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:45.747766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:45.748360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:45.748638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:45.748908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:45.749541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:45.749843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:45.751235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:45.762712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 639/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Sep 12 11:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:45.768318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.865052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1c88c-95fc-4714-9f8f-94e29230393f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:45.878479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:45.898502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:45.899150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:45.899434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:45.899712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:45.906178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:45.907644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 11:43:45.918848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:45.919725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: ERROR keystone.server.flask.application Sep 12 11:43:45.920709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 646/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Sep 12 11:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:45.925886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:45.975145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:46.023890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337f6bc-877b-4acd-abba-8ba356507736 None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:46.055681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:46.057083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:46.057552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:46.057995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:46.059180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:46.059666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:46.061939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b62e0-b085-48ec-8e13-f92d5688289f None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:46.080794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 640/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Sep 12 11:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:46.087080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:46.184514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640290-7019-4f4e-ad59-6b5ad74c7a8f None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:46.217377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:46.217970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:46.218248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:46.218547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:46.219266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] RBAC: Adding query filter params (role.id=ada491db6331406099f2a486b26a3cc4, scope.project.id=77204d1f5d484097a21762f2a6bef64c, user.id=27b96a89a0d140aeaeba8078a21b4cbf) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:46.219572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:46.220977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f81d75-783a-48f2-8730-37f2eb6330ca None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:46.226537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 647/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Sep 12 11:43:46 2022] GET /identity/v3/role_assignments?user.id=27b96a89a0d140aeaeba8078a21b4cbf&scope.project.id=77204d1f5d484097a21762f2a6bef64c&role.id=ada491db6331406099f2a486b26a3cc4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:48.363524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05f43-85ac-48e0-8262-94657f48f51f None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:48.364043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05f43-85ac-48e0-8262-94657f48f51f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:48.364043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af05f43-85ac-48e0-8262-94657f48f51f None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:48.365457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 641/1288] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:48.370498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:48.370902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:48.371105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:48.388951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:48.390119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:48.419388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:48.455573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 648/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:48.727387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af05f43-85ac-48e0-8262-94657f48f51f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:48.839736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af05f43-85ac-48e0-8262-94657f48f51f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:48.849864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:48.873010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:48.873627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:48.873902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:48.874170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:48.874777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:48.876134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:48.885257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:48.886483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:48.887634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 642/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:43:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:48.892266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:48.986400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d91cea-86e2-4211-b076-e1d404f9b502 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:49.019626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:49.020217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:49.020493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:49.020762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:49.021380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:49.021679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:49.023049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d17166-466a-4ae4-a73f-fe1c8f946ef0 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:49.027832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 649/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 12 11:43:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:49.033605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:49.131848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73e02-4b00-44d5-9e65-6f24cd2dc5ac None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:49.163694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:49.164282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:49.164555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:49.164861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] PATH_INFO: `/v3/services` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:49.165665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:49.165966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:49.167350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bfe58-39ae-4651-8579-42e0c1d757c0 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:49.171575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 643/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 12 11:43:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:51.312086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41359-8897-4d4e-a26a-bd108eb8e05f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:51.312632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41359-8897-4d4e-a26a-bd108eb8e05f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:51.312632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41359-8897-4d4e-a26a-bd108eb8e05f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:51.313970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 650/1293] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:51.319213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961abb6-3777-4330-a60d-3da6fef8ed4b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:51.319579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961abb6-3777-4330-a60d-3da6fef8ed4b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:51.319767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961abb6-3777-4330-a60d-3da6fef8ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:51.338536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a961abb6-3777-4330-a60d-3da6fef8ed4b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:51.339724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a961abb6-3777-4330-a60d-3da6fef8ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:51.367495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:51.404343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 644/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:51.676686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b41359-8897-4d4e-a26a-bd108eb8e05f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:51.708215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:51.778607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b41359-8897-4d4e-a26a-bd108eb8e05f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:51.810960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:51.811555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:51.811832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:51.812105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:51.812759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:51.814112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf7e70-4492-4e89-ae1e-c93c08f15a8f None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:51.828778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 651/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Sep 12 11:43:51 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:43:54.043488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78eaac-89e5-45ba-a838-4f1dc064edb8 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.044019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78eaac-89e5-45ba-a838-4f1dc064edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.044019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78eaac-89e5-45ba-a838-4f1dc064edb8 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.045369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 645/1296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:54.050549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.050976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.051069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.068796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:54.069964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:54.143434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 652/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:54.409655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78eaac-89e5-45ba-a838-4f1dc064edb8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:54.507639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78eaac-89e5-45ba-a838-4f1dc064edb8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:54.540031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:54.540701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.540978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.541246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.541816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:54.543193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:54.551615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:54.552830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:54.553986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 646/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:43:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:54.559498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:54.577440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:54.655626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6c84-efad-4d2b-9a26-3680653a5cc2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:54.688444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:54.688977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.689251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.689517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.690132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:54.690428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:54.691796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:54.696793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 653/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 12 11:43:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:54.702854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:54.738760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:54.810677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9772c3-538d-466b-8d2d-447d2e4b19dc None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:54.844934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:54.845554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.845831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.846102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.846826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] RBAC: Adding query filter params (interface=public, service_id=7a866a10726e406b9acc58736fc19271, region_id=RegionOne) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:54.847133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:54.848495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65876a46-4abb-4453-9442-a88f940e4b65 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:54.853552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 647/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Sep 12 11:43:54 2022] GET /identity/v3/endpoints?service_id=7a866a10726e406b9acc58736fc19271&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:43:54.859293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:54.937784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:54.956364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ad18-d287-41c4-8141-dd4d071b897d None admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:54.988903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:54.989500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:54.989779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:54.990056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:54.990687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:54.992037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f998c-567a-4209-b36a-ec42b95cb5c1 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:54.999200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 654/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Sep 12 11:43:54 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:43:57.118049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eb43-9c20-4ac3-9cb2-08436669d99d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:57.118453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eb43-9c20-4ac3-9cb2-08436669d99d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:57.118623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eb43-9c20-4ac3-9cb2-08436669d99d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:57.119978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 648/1302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:43:57.125195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:57.125628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:57.125733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:57.144165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:43:57.145346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:43:57.210085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 655/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:43:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 11:43:57.477541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3eb43-9c20-4ac3-9cb2-08436669d99d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:57.569566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:57.577463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3eb43-9c20-4ac3-9cb2-08436669d99d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:57.610023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:57.610667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:57.610947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:57.611216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] PATH_INFO: `/v3/services/placement` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:57.611812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:57.613176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 11:43:57.622388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 11:43:57.623672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application with Lock( Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self._enter() Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application return self.creator() Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application created_value = creator( Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: ERROR keystone.server.flask.application Sep 12 11:43:57.624728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 649/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Sep 12 11:43:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 11:43:57.629375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:57.702910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:43:57.724786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186453-b256-4c4f-97a7-d77dde5151ba None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:57.758558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:57.759116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:57.759393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:57.759664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] PATH_INFO: `/v3/services` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:57.760285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:43:57.760584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:57.761942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca9ee7-ca3f-4429-9b02-9f551592ed27 None admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:57.768327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 656/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Sep 12 11:43:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:43:57.774161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:43:57.870378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f260-f163-4143-a4b0-ac74e03783af None admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:43:57.903981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:43:57.904571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:43:57.904848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:43:57.905115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:43:57.905765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:43:57.907146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9e4d-ad43-4943-ac35-da2a881c7912 None admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:43:57.926335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 650/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:43:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:44:07.764192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=30 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:44:07.922176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:02.173656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d0dc-22df-4e18-bd39-59b703d64711 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:02.174849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d0dc-22df-4e18-bd39-59b703d64711 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:02.175366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d0dc-22df-4e18-bd39-59b703d64711 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:02.177228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 657/1307] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:02.182466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb807f-da00-491e-a0aa-a3d24bf3c419 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:02.183335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb807f-da00-491e-a0aa-a3d24bf3c419 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:02.183611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb807f-da00-491e-a0aa-a3d24bf3c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:02.278116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-53cb807f-da00-491e-a0aa-a3d24bf3c419 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:02.279341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-53cb807f-da00-491e-a0aa-a3d24bf3c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:02.382551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 651/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:02.682761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:02.683113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:02.683336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:02.684736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 658/1309] 10.222.0.15 () {54 vars in 965 bytes} [Mon Sep 12 11:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:02.692096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4b5f-e6ce-4b63-9787-f12c0e5f4204 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:02.692424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4b5f-e6ce-4b63-9787-f12c0e5f4204 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:02.692658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4b5f-e6ce-4b63-9787-f12c0e5f4204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:02.777754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b5fd4b5f-e6ce-4b63-9787-f12c0e5f4204 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:02.778983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b5fd4b5f-e6ce-4b63-9787-f12c0e5f4204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:02.800254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:02.860475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 652/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 12 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:02.870257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:02.872096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:02.999893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69f8e2-4cb8-4224-8e50-e289cd463b05 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:03.089160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:03.089780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:03.090114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:03.090384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:03.092075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:03.154231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:03.215104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:03.216538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5d3c-f885-40f3-bdcc-6bd696ed1ab0 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:03.237107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 659/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:05.712127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:05.712645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:05.712727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:05.714181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 653/1312] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:05.719341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae24-f420-43cb-92c1-56490611b360 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:05.719750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae24-f420-43cb-92c1-56490611b360 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:05.719893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae24-f420-43cb-92c1-56490611b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:05.809789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-01c6ae24-f420-43cb-92c1-56490611b360 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:05.811062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-01c6ae24-f420-43cb-92c1-56490611b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:05.914723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 660/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:06.196860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:06.198712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:06.305110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6d4b7-a499-4cac-9ec9-75f3e37b6a6c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:06.376671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:06.379441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:06.380145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:06.380668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:06.380993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:06.383006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:06.491573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:06.493128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d686bd-efe9-41bf-89a9-ea7b3d2cf30b service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:06.512550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 654/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:09.001387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:09.002412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:09.002592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:09.004125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 661/1315] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:09.009404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03e98-b6c5-4c51-b133-83deff0ec03a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:09.009833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03e98-b6c5-4c51-b133-83deff0ec03a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:09.009951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03e98-b6c5-4c51-b133-83deff0ec03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:09.074827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b2f03e98-b6c5-4c51-b133-83deff0ec03a None None] MFA Rules not processed for user `27c6e1813c56499ab5b07dc305f8bdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:09.076010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b2f03e98-b6c5-4c51-b133-83deff0ec03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:09.129362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:09.173304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 655/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:09.459929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:09.461491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:09.471938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:09.567930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b773-6060-47f0-ba2f-11dc9dc1ec03 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:09.645573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:09.646178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:09.646454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:09.646750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:09.648342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:09.710248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:09.757142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:09.758619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce149e-3426-4f4d-a2d5-2413bb60e235 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:09.777336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 662/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:12.520633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5794d1-b1bf-430a-ad71-e0b70d2c95ec None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:12.521157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5794d1-b1bf-430a-ad71-e0b70d2c95ec None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:12.521157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5794d1-b1bf-430a-ad71-e0b70d2c95ec None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:12.522463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 656/1318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:12.527750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5636f-051f-412c-ad49-2a795a2c702d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:12.528157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5636f-051f-412c-ad49-2a795a2c702d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:12.528305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5636f-051f-412c-ad49-2a795a2c702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:12.593371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-94c5636f-051f-412c-ad49-2a795a2c702d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:12.594614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-94c5636f-051f-412c-ad49-2a795a2c702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:12.691522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 663/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:13.000157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:13.000444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:13.000716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:13.002066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 657/1320] 10.222.0.15 () {54 vars in 965 bytes} [Mon Sep 12 11:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:13.008527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6b46-e2d4-4a57-b63b-924603703d67 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:13.008837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6b46-e2d4-4a57-b63b-924603703d67 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:13.009114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d6b46-e2d4-4a57-b63b-924603703d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:13.031190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:13.085704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-575d6b46-e2d4-4a57-b63b-924603703d67 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:13.086998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-575d6b46-e2d4-4a57-b63b-924603703d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:13.163062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 664/1321] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Sep 12 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:13.171917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:13.173444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:13.277931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cac8c-aad0-4ef7-8031-6ffb98125c11 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:13.350970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:13.351573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:13.351850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:13.352120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:13.353708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:13.360453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:13.461979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:13.463432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84c07c-1834-4709-8306-8f122d6a11a1 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:13.481798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 658/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:16.756614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:16.757187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:16.757187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:16.758558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 665/1323] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:16.764492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb017f-9f68-4ef2-a4b7-67344f46a395 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:16.765065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb017f-9f68-4ef2-a4b7-67344f46a395 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:16.765065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb017f-9f68-4ef2-a4b7-67344f46a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:16.840303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-16bb017f-9f68-4ef2-a4b7-67344f46a395 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:16.841569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-16bb017f-9f68-4ef2-a4b7-67344f46a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:16.904686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:16.945629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 659/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:17.228680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:17.230243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:17.336794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08312-7ce6-4a3b-a5fa-66ddedd6f22f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:17.349587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:17.411242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:17.411765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:17.412042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:17.412310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:17.413930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:17.524660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:17.526226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9f7a9-57ca-4209-9f40-a0c378d5e1e0 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:17.545562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 666/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:19.762310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:19.762775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:19.762928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:19.764390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 660/1326] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:19.769839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987015a-4dee-463c-8f47-9714c9e09f62 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:19.770280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987015a-4dee-463c-8f47-9714c9e09f62 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:19.770548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987015a-4dee-463c-8f47-9714c9e09f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:19.826261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:19.845639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9987015a-4dee-463c-8f47-9714c9e09f62 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:19.847149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9987015a-4dee-463c-8f47-9714c9e09f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:19.954188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 667/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:20.233410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:20.235032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:20.341112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a1e79-ab19-4c31-aa09-b6d3004c3d26 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:20.414300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:20.414854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:20.415126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:20.415392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:20.416962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:20.433489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:20.525799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:20.527285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebc4f4-abb1-486c-9ee3-1cb35c8a486c service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:20.545517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 661/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:22.751397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:22.751891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:22.752041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:22.754017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 668/1329] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:22.759587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cea84-c40a-4bf5-9fc9-0101ac379a25 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:22.759987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cea84-c40a-4bf5-9fc9-0101ac379a25 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:22.760130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cea84-c40a-4bf5-9fc9-0101ac379a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:22.827303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-512cea84-c40a-4bf5-9fc9-0101ac379a25 None None] MFA Rules not processed for user `27c6e1813c56499ab5b07dc305f8bdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:22.828423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-512cea84-c40a-4bf5-9fc9-0101ac379a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:22.891941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:22.925438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 662/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:23.209084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:23.210686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:23.319784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41410b5f-1a00-4822-b96b-c430a1d02ea1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:23.340593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:23.394256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:23.394891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:23.395173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:23.395448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:23.397052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:23.508877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:23.510323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb60c5e-a1ca-4aaa-8d4e-4c5478fccc6e service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:23.529758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 669/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:25.921180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:25.921758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:25.921758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:25.923109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 663/1332] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:25.929256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db32-3223-47ee-9b60-bd8ac7e3615d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:25.929691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db32-3223-47ee-9b60-bd8ac7e3615d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:25.929808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db32-3223-47ee-9b60-bd8ac7e3615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:25.995483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:25.998128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-65a8db32-3223-47ee-9b60-bd8ac7e3615d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:25.999939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-65a8db32-3223-47ee-9b60-bd8ac7e3615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:26.095863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 670/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:26.380512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:26.382081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:26.484181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794060e-c0b2-4f8e-aa3a-23151294a272 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:26.559829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:26.560429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:26.560708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:26.560978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:26.562639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:26.590494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:26.675860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:26.677232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa086b0-68af-4347-bed0-c1d9201bed78 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:26.695893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 664/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:28.880156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:28.880740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:28.880740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:28.882085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 671/1335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:28.887218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460a74-a51b-4d17-8f58-b6a4818b309d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:28.887606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460a74-a51b-4d17-8f58-b6a4818b309d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:28.887764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c460a74-a51b-4d17-8f58-b6a4818b309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:28.957099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2c460a74-a51b-4d17-8f58-b6a4818b309d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:28.958316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2c460a74-a51b-4d17-8f58-b6a4818b309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:29.031795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:29.054903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 665/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:29.341476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:29.343112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:29.445531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6850d-5905-451f-aa92-71b7bd1eacb1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:29.474761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:29.519507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:29.520103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:29.520380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:29.520648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:29.522233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:29.629385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:29.630821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31697be4-8753-4c5a-bf40-546cc91473cb service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:29.648998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 672/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:32.117962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:32.118416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:32.118544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:32.119873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 666/1338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:45:32.124983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade53c9f-afc8-4cdb-b3b7-e4e2600dedb8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:32.125425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade53c9f-afc8-4cdb-b3b7-e4e2600dedb8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:32.125539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade53c9f-afc8-4cdb-b3b7-e4e2600dedb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:32.201844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ade53c9f-afc8-4cdb-b3b7-e4e2600dedb8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:32.203101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ade53c9f-afc8-4cdb-b3b7-e4e2600dedb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:32.212244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:32.299865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 673/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:32.583672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:32.585220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:32.689587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4742-1bb9-4d75-98d8-1d4bab64f58e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:32.763223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:32.763827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:32.764104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:32.764372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:32.765960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:32.800522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:32.872577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:32.874015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311fee-4415-4a29-a7d0-0f6a06654b9d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:32.892989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 667/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:36.940522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412775-d363-4e48-98c9-95a09567941f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:36.941052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412775-d363-4e48-98c9-95a09567941f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:36.941052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412775-d363-4e48-98c9-95a09567941f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:36.942576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 674/1341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:36.947936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4083ce-a52a-4340-9e12-13cd57f6e5af None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:36.948348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4083ce-a52a-4340-9e12-13cd57f6e5af None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:36.948497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4083ce-a52a-4340-9e12-13cd57f6e5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:37.016014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7e4083ce-a52a-4340-9e12-13cd57f6e5af None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:37.017465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7e4083ce-a52a-4340-9e12-13cd57f6e5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:37.118010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:37.142706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 668/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:37.415836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412775-d363-4e48-98c9-95a09567941f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:37.417382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f4412775-d363-4e48-98c9-95a09567941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:37.528050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412775-d363-4e48-98c9-95a09567941f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:37.567545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:37.603877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:37.604484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:37.604759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:37.605032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:37.606644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:37.719278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:37.720618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a8140-6ae8-4630-a422-17ad3c7f8c42 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:37.740538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 675/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:40.556180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:40.556706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:40.556706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:40.557971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 669/1344] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:40.564945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145cc0-c15e-49a0-bdac-9bac007596bf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:40.565390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145cc0-c15e-49a0-bdac-9bac007596bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:40.565644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145cc0-c15e-49a0-bdac-9bac007596bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:40.633136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-73145cc0-c15e-49a0-bdac-9bac007596bf None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:40.634626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-73145cc0-c15e-49a0-bdac-9bac007596bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:40.653379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:40.736843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 676/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:45:41.024928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:41.026456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:41.134351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcff41c-9aa5-4639-970e-99e04e5ceada None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:41.212535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:41.213145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:41.213425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:41.213688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:41.215339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:41.265617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:41.334078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:41.335572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6d94a-236d-48db-b0aa-a75ca4df95a8 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:41.354315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 670/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:45.901032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:45.901541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:45.901637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:45.903230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 677/1347] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:45.908389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92728-6d93-4e63-89f0-70c115e1bbac None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:45.908825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92728-6d93-4e63-89f0-70c115e1bbac None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:45.908969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92728-6d93-4e63-89f0-70c115e1bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:45.982531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-82f92728-6d93-4e63-89f0-70c115e1bbac None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:45.984007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-82f92728-6d93-4e63-89f0-70c115e1bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:46.089762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:46.092403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 671/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:46.372583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:45:46.374131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:46.476328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a1703-402a-4587-9f16-5bca77330f3d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:45:46.524573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:46.550078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:45:46.550701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:46.550984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:46.551250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:46.552838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:46.661375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:45:46.662731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47efecb-8e81-4674-a3d4-e408d607cb34 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:45:46.680567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 678/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:45:51.577323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6475fe-7241-48dc-91d1-70d6ef4c40c7 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:51.577844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6475fe-7241-48dc-91d1-70d6ef4c40c7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:51.578120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6475fe-7241-48dc-91d1-70d6ef4c40c7 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:51.579489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 672/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:51.584951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56730b8d-94de-4709-a8c1-2d4cf8097cc9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:51.585331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56730b8d-94de-4709-a8c1-2d4cf8097cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:51.585507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56730b8d-94de-4709-a8c1-2d4cf8097cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:51.754886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-56730b8d-94de-4709-a8c1-2d4cf8097cc9 None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:51.756057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-56730b8d-94de-4709-a8c1-2d4cf8097cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:51.791680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:51.892787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 679/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:56.009450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8593e9-db48-4ed3-8884-12f0d90fe85a None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:56.009922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8593e9-db48-4ed3-8884-12f0d90fe85a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:56.010033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8593e9-db48-4ed3-8884-12f0d90fe85a None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:56.011541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 673/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:56.016638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7cae5-ef8b-490b-b69a-7210402696b6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:56.017465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7cae5-ef8b-490b-b69a-7210402696b6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:56.017818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7cae5-ef8b-490b-b69a-7210402696b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:56.106661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a0c7cae5-ef8b-490b-b69a-7210402696b6 None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:56.107890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a0c7cae5-ef8b-490b-b69a-7210402696b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:56.182928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:45:56.222345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 680/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:45:57.655143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0c63-9261-4735-bffb-54309b959b05 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:57.655795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0c63-9261-4735-bffb-54309b959b05 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:57.655795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0c63-9261-4735-bffb-54309b959b05 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:57.657158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 674/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:45:57.662022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e500-c6d1-4274-ad20-50938c79b1aa None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:45:57.662808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e500-c6d1-4274-ad20-50938c79b1aa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:45:57.663085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e500-c6d1-4274-ad20-50938c79b1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:45:57.743664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2150e500-c6d1-4274-ad20-50938c79b1aa None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:45:57.744872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2150e500-c6d1-4274-ad20-50938c79b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:45:57.855589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 681/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:45:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:46:01.379143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2f78c-5637-42c2-8191-97e04be9e4c6 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:01.380345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2f78c-5637-42c2-8191-97e04be9e4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:01.380848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2f78c-5637-42c2-8191-97e04be9e4c6 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:01.387196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 675/1356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:01.388115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f11f2-549e-41dc-ba91-fb71d4c5d12c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:01.388547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f11f2-549e-41dc-ba91-fb71d4c5d12c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:01.388664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f11f2-549e-41dc-ba91-fb71d4c5d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:01.430862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:01.465779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-805f11f2-549e-41dc-ba91-fb71d4c5d12c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:01.467007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-805f11f2-549e-41dc-ba91-fb71d4c5d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:01.568734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 682/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:01.873187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:01.874096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:01.874589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:01.876096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 676/1358] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 12 11:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:01.884508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab9d2-8183-4413-acc4-daa3bd171272 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:01.884936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab9d2-8183-4413-acc4-daa3bd171272 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:01.885443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab9d2-8183-4413-acc4-daa3bd171272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:01.976429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b17ab9d2-8183-4413-acc4-daa3bd171272 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:01.977716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b17ab9d2-8183-4413-acc4-daa3bd171272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:02.029525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:02.103449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 683/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 12 11:46:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:02.114246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:02.115794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:02.279162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71601f7-805a-4ccc-a54b-1984d15910f0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:02.359170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:02.401107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:02.401785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:02.402031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:02.402258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:02.403877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:02.508880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:02.510498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb96bf-03ba-43c6-8e31-253caf6dcc41 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:02.531952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 677/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:04.924983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c431-d1db-41bb-833e-ca642889967e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:04.925536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c431-d1db-41bb-833e-ca642889967e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:04.925536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072c431-d1db-41bb-833e-ca642889967e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:04.926827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 684/1361] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:04.932249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edac9e-f127-488c-af08-06a90e9ea2e7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:04.932685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edac9e-f127-488c-af08-06a90e9ea2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:04.932799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edac9e-f127-488c-af08-06a90e9ea2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:05.003085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-38edac9e-f127-488c-af08-06a90e9ea2e7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:05.004271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-38edac9e-f127-488c-af08-06a90e9ea2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:05.042924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:05.113703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 678/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:05.401143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072c431-d1db-41bb-833e-ca642889967e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:05.403107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3072c431-d1db-41bb-833e-ca642889967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:05.592670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072c431-d1db-41bb-833e-ca642889967e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:05.604814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:05.724472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:05.725507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:05.726573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:05.726857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:05.728973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:05.828720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:05.830346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f55a40-3770-4b30-8510-760ba65d8ceb service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:05.848934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 685/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:08.375640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5ae0-4bfa-46b7-bc35-9d8fd940a651 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:08.376195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5ae0-4bfa-46b7-bc35-9d8fd940a651 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:08.376195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5ae0-4bfa-46b7-bc35-9d8fd940a651 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:08.377526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 679/1364] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:08.382957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352183-de65-4506-b3b8-db595baf6f84 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:08.383358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352183-de65-4506-b3b8-db595baf6f84 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:08.383512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352183-de65-4506-b3b8-db595baf6f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:08.400517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:08.468255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a8352183-de65-4506-b3b8-db595baf6f84 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:08.469475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a8352183-de65-4506-b3b8-db595baf6f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:08.574397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 686/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:08.868902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:08.869336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:08.869708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:08.871407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 680/1366] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 12 11:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:08.877219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0dc1-dfa1-4ae2-8452-8506fc5b6dcc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:08.877511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0dc1-dfa1-4ae2-8452-8506fc5b6dcc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:08.877790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0dc1-dfa1-4ae2-8452-8506fc5b6dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:08.958729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e11b0dc1-dfa1-4ae2-8452-8506fc5b6dcc None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:08.960104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e11b0dc1-dfa1-4ae2-8452-8506fc5b6dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:08.998701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:09.094464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 687/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 12 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:09.102794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:09.104568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:09.274577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281d454-5f70-4c38-bc42-9d425ffc5557 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:09.296712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:09.398758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:09.399416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:09.399649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:09.399867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:09.401492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:09.511813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:09.514156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463465a-ce6a-4734-88d9-acf7ffe9f875 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:09.534023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 681/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:09.591137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca9673-2cf0-462a-873d-832807feefae None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:09.592173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca9673-2cf0-462a-873d-832807feefae None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:09.592668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca9673-2cf0-462a-873d-832807feefae None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:09.594869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 688/1369] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:09.599985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c5c93-b8a5-415d-819c-79ed15130ca1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:09.600275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c5c93-b8a5-415d-819c-79ed15130ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:09.600673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c5c93-b8a5-415d-819c-79ed15130ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:09.635435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:09.678455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5b1c5c93-b8a5-415d-819c-79ed15130ca1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:09.679643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5b1c5c93-b8a5-415d-819c-79ed15130ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:09.804752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 682/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:10.139884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:10.139884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:10.139884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:10.139884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 689/1371] 10.222.0.15 () {54 vars in 964 bytes} [Mon Sep 12 11:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:10.144380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1de8-cf87-4230-add1-5a978d4050b2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:10.144784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1de8-cf87-4230-add1-5a978d4050b2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:10.145154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c1de8-cf87-4230-add1-5a978d4050b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:10.233620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ec3c1de8-cf87-4230-add1-5a978d4050b2 None None] MFA Rules not processed for user `6d11c4cfca65434c9108393d11c1f7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:10.235043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ec3c1de8-cf87-4230-add1-5a978d4050b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:10.291418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:10.327513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 683/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Sep 12 11:46:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:10.336124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:10.337904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:10.455971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b7001-7ad0-48f4-9d68-6df52ab6f966 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:10.503116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:10.548188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:10.548944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:10.549062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:10.549342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:10.551050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:10.685874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:10.687342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e2a56-e8a5-4b28-928f-0d69ebde038c service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:10.708738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 690/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:11.786161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:11.786747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:11.786747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:11.788140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 684/1374] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:11.793908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ee580-a7a9-4c8c-8d6d-027c9c8b8998 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:11.794305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ee580-a7a9-4c8c-8d6d-027c9c8b8998 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:11.794822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ee580-a7a9-4c8c-8d6d-027c9c8b8998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:11.863437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-731ee580-a7a9-4c8c-8d6d-027c9c8b8998 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:11.864651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-731ee580-a7a9-4c8c-8d6d-027c9c8b8998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:11.886771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:11.968926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 691/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:12.260507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:12.262157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:12.422784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e1805-1014-41db-89d0-6d3fa9b7134c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:12.480347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:12.547872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:12.548480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:12.548760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:12.549028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:12.550694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:12.646149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:12.647459 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f0c33-8f96-413a-af17-8af72cdace31 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:12.666443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 685/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:14.892437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:14.893012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:14.893012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:14.894691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 692/1377] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:14.899870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563248-16e6-4600-be08-9d72d387797f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:14.900257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563248-16e6-4600-be08-9d72d387797f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:14.900434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7563248-16e6-4600-be08-9d72d387797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:14.970553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f7563248-16e6-4600-be08-9d72d387797f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:14.971827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f7563248-16e6-4600-be08-9d72d387797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:14.979507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:15.090031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 686/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:15.382479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:15.384239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:15.538174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc09de-0c97-4050-8fe3-262a69ca136c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:15.542306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:15.656540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:15.657229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:15.657603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:15.657973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:15.659695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:15.754890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:15.756655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ffcb-8bf8-436b-b432-fadfe62c01d8 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:15.776019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 693/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:18.050641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:18.051152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:18.051152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:18.052611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 687/1380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:18.057657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20763f-191c-4dc7-8292-9c834eaee58c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:18.058047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20763f-191c-4dc7-8292-9c834eaee58c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:18.058238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20763f-191c-4dc7-8292-9c834eaee58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:18.070843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:18.140874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7c20763f-191c-4dc7-8292-9c834eaee58c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:18.142066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7c20763f-191c-4dc7-8292-9c834eaee58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:18.281229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 694/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:18.561213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:18.562712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:18.703330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:18.714397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb8259-7694-4fa3-85e4-19db5afd7bd7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:18.839677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:18.840208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:18.840483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:18.841416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:18.843081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:18.936001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:18.937596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fa118-bbb3-4368-a594-c5c829fd00f1 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:18.948206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:18.958191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 688/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:46:21.169847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:21.170360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:21.170604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:21.172441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 695/1383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:21.177633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400b31-0fbc-4156-844b-302f3ff8a19a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:21.178083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400b31-0fbc-4156-844b-302f3ff8a19a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:21.178178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400b31-0fbc-4156-844b-302f3ff8a19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:21.245554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-54400b31-0fbc-4156-844b-302f3ff8a19a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:21.246819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-54400b31-0fbc-4156-844b-302f3ff8a19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:21.347265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 689/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:21.634606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:21.636695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:21.739102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:21.803233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2681ab-eed6-4366-af77-e6eccc79805e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:21.942190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:21.942991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:21.943361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:21.944082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:21.946263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:22.015252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:22.057107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:22.058753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e88a12-4fd1-4c2a-88be-c07e1ffc2873 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:22.077624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 696/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:24.287890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:24.288465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:24.288465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:24.289785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 690/1386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:24.295675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459b72-b3a3-4918-98ce-22bc7be17eca None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:24.296099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459b72-b3a3-4918-98ce-22bc7be17eca None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:24.296247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3459b72-b3a3-4918-98ce-22bc7be17eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:24.362903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c3459b72-b3a3-4918-98ce-22bc7be17eca None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:24.364130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c3459b72-b3a3-4918-98ce-22bc7be17eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:24.463784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 697/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:24.752350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:24.753975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:24.833698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:24.911977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da215-e335-49e8-88ac-e30e476c3035 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:25.026993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:25.027512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:25.027830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:25.028095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:25.029689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:25.068986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:25.124514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:25.126123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14822c-91b8-4f7c-81e8-82c92254ab3f service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:25.144534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 691/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:26.120065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22bbb-ed6d-4b22-ad64-79568c85b079 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:26.120495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22bbb-ed6d-4b22-ad64-79568c85b079 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:26.120647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22bbb-ed6d-4b22-ad64-79568c85b079 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:26.122795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 698/1389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:26.128165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39f2a-5223-41ab-8f0b-44ba1e637b77 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:26.128581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39f2a-5223-41ab-8f0b-44ba1e637b77 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:26.128740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39f2a-5223-41ab-8f0b-44ba1e637b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:26.195742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-58e39f2a-5223-41ab-8f0b-44ba1e637b77 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:26.196941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-58e39f2a-5223-41ab-8f0b-44ba1e637b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:26.294145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 692/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:26.592721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ac41-1152-4132-982f-948fde32850a None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:26.593002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ac41-1152-4132-982f-948fde32850a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:26.593277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ac41-1152-4132-982f-948fde32850a None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:26.594636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 699/1391] 10.222.0.15 () {54 vars in 966 bytes} [Mon Sep 12 11:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:26.599770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8732-c039-4b99-86a4-8a6a76d2de6b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:26.600127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8732-c039-4b99-86a4-8a6a76d2de6b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:26.600475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8732-c039-4b99-86a4-8a6a76d2de6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:26.616576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:26.677547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-703a8732-c039-4b99-86a4-8a6a76d2de6b None None] MFA Rules not processed for user `02a59e7ff79249eaa7f45cdbe15d0eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:26.678757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-703a8732-c039-4b99-86a4-8a6a76d2de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:26.754918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 693/1392] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Sep 12 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:26.764656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1ac41-1152-4132-982f-948fde32850a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:26.766350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-99c1ac41-1152-4132-982f-948fde32850a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:26.797587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:26.873443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1ac41-1152-4132-982f-948fde32850a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:26.950481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:26.951118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:26.951401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:26.951673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:26.953298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:27.034652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:27.067934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:27.069625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27a6e9-fa75-49e0-b80e-80d4e2f38434 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:27.089806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 700/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:27.375556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776365-333c-4f6d-baea-1b851f05a5b8 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:27.376148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776365-333c-4f6d-baea-1b851f05a5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:27.376148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776365-333c-4f6d-baea-1b851f05a5b8 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:27.377496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 694/1394] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:27.383307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:27.383763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:27.383859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:27.459471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:27.460701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:27.583673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 701/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:27.870432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4a65-e43f-4221-8fd2-95963bda522a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:27.870766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4a65-e43f-4221-8fd2-95963bda522a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:27.871028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4a65-e43f-4221-8fd2-95963bda522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:27.887591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:27.889114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:27.931123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:27.940904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7c0d4a65-e43f-4221-8fd2-95963bda522a None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:27.942102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7c0d4a65-e43f-4221-8fd2-95963bda522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:27.999699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:28.010823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30de0-200e-4b4c-bc93-96d9aca6eb9a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:28.033541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 695/1396] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:28.080725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2416e2-2c37-4f9d-906e-0691b750fbbc None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:28.081012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2416e2-2c37-4f9d-906e-0691b750fbbc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:28.081288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2416e2-2c37-4f9d-906e-0691b750fbbc None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:28.082881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 696/1397] 10.222.0.15 () {54 vars in 943 bytes} [Mon Sep 12 11:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:46:28.093190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f34f2-ce22-471b-9425-db9973679478 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:28.093551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f34f2-ce22-471b-9425-db9973679478 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:28.093898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f34f2-ce22-471b-9425-db9973679478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:28.132319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:28.132950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:28.133231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:28.133516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:28.135174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:28.172324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:28.177693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2b5f34f2-ce22-471b-9425-db9973679478 None None] MFA Rules not processed for user `091613b4778d4946b0059334bdb70dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:28.178915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2b5f34f2-ce22-471b-9425-db9973679478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:28.232808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:28.234551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:28.253321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 702/1398] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:46:28.299271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 697/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Sep 12 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:46:28.309441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:28.310711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:28.419566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fcad2-2095-4c77-849f-5903165c5b46 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:28.498680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:28.539364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:28.539979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:28.540267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:28.540537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:28.542180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:28.627306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:28.628896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d13ce4-3bfc-459b-a6fe-b01ed4cb0d7c service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:28.637858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 703/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:46:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:46:30.479870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:30.480413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:30.480413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:30.481918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 698/1401] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 11:46:30.487404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680602c7-d1f6-4904-a3dd-a2d1b97b9365 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:30.487810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680602c7-d1f6-4904-a3dd-a2d1b97b9365 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:30.487955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680602c7-d1f6-4904-a3dd-a2d1b97b9365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:30.563588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-680602c7-d1f6-4904-a3dd-a2d1b97b9365 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:30.564814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-680602c7-d1f6-4904-a3dd-a2d1b97b9365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:30.626695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:30.675606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 704/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:30.983041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:30.984591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:31.022406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:31.153188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637c9cd-c1b0-49f7-8e03-8f5c248ff4db None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:31.271113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:31.283249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:31.283868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:31.284145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:31.284415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:31.286032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:31.385916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:31.387605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0bcb0-c279-4c97-b89a-1bf4d3d8e2ee service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:31.407170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 699/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:33.615965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:33.615965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:33.615965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:33.616748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 705/1404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:33.621991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560716-e042-449b-8454-23b16b64c207 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:33.622447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560716-e042-449b-8454-23b16b64c207 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:33.622582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560716-e042-449b-8454-23b16b64c207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:33.691316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b2560716-e042-449b-8454-23b16b64c207 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:33.692537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b2560716-e042-449b-8454-23b16b64c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:33.760961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:33.800896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 700/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:34.097948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:34.099538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:34.288864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3790c9-c788-4aff-9e5b-518d37f2f9b2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:34.332376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:34.425132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:34.425741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:34.426023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:34.426296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:34.427966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:34.530862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:34.532466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbe5b8-0d00-483b-845f-1505a47f4cb2 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:34.553446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 706/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:36.845091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:36.845640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:36.845640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:36.852385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7b6e3-5ad0-4360-af46-d4b22b5b8202 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:36.853171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7b6e3-5ad0-4360-af46-d4b22b5b8202 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:36.853451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7b6e3-5ad0-4360-af46-d4b22b5b8202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:36.858573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 701/1407] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:46:36 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:36.912610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:36.929072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3dd7b6e3-5ad0-4360-af46-d4b22b5b8202 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:36.930295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3dd7b6e3-5ad0-4360-af46-d4b22b5b8202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:37.037616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 707/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:37.343539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:37.345095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:37.550121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684007f8-becd-4f81-8885-e24dbc3c9b87 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:37.622272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:37.702680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112187-ee61-428a-a63c-f648a994790c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:37.703349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112187-ee61-428a-a63c-f648a994790c service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:37.703718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112187-ee61-428a-a63c-f648a994790c service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:37.704332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112187-ee61-428a-a63c-f648a994790c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:37.706009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-45112187-ee61-428a-a63c-f648a994790c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:37.804166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112187-ee61-428a-a63c-f648a994790c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:37.805781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112187-ee61-428a-a63c-f648a994790c service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:37.825578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 702/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:47.036028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=46 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:47.822738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=76 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:51.600895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8b6-ddda-4e36-b3d9-e9b94c0df82f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:51.601444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8b6-ddda-4e36-b3d9-e9b94c0df82f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:51.601444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8b6-ddda-4e36-b3d9-e9b94c0df82f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:51.602829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 708/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Sep 12 11:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:51.608232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b9a8-acf5-4a9e-b5df-fa6a52e31a25 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:51.608651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b9a8-acf5-4a9e-b5df-fa6a52e31a25 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:51.608801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b9a8-acf5-4a9e-b5df-fa6a52e31a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:51.684604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fdb3b9a8-acf5-4a9e-b5df-fa6a52e31a25 None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:51.685795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fdb3b9a8-acf5-4a9e-b5df-fa6a52e31a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:51.791105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 703/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 12 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:51.805800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c13e-44a1-43c3-b968-0d8196a82538 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:51.806089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c13e-44a1-43c3-b968-0d8196a82538 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:51.806366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c13e-44a1-43c3-b968-0d8196a82538 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:51.807726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 709/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Sep 12 11:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:51.812354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878727a-ec8f-4c3b-8603-83466e793797 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:51.812643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878727a-ec8f-4c3b-8603-83466e793797 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:51.812910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878727a-ec8f-4c3b-8603-83466e793797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:51.886192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:51.890159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0878727a-ec8f-4c3b-8603-83466e793797 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:51.891831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0878727a-ec8f-4c3b-8603-83466e793797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:52.005388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 704/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 12 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:54.587140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:54.587662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:54.587662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:54.588944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:54.594792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de83d-8964-495a-9779-90f63d413901 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:54.595173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de83d-8964-495a-9779-90f63d413901 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:54.595347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de83d-8964-495a-9779-90f63d413901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:54.661517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-039de83d-8964-495a-9779-90f63d413901 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:54.662747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-039de83d-8964-495a-9779-90f63d413901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:54.717757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:54.760918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:55.055695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:55.057427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:55.208590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e1acf-e89c-44d8-8bb5-b225cc20e545 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:55.280684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:55.320867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:55.321407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:55.321678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:55.321942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:55.323662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:55.412567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:55.414164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38823ea1-4495-4604-9f3b-ca22386e53e4 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:55.432546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 711/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:56.564107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904df280-f766-4566-8444-2552ed870f6a None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:56.564623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904df280-f766-4566-8444-2552ed870f6a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:56.564623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904df280-f766-4566-8444-2552ed870f6a None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:56.565941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:56.572443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1ed5-03f7-4f67-b04b-7b265beafee4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:56.572878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1ed5-03f7-4f67-b04b-7b265beafee4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:56.572996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1ed5-03f7-4f67-b04b-7b265beafee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:56.649065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-60fb1ed5-03f7-4f67-b04b-7b265beafee4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:56.650267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-60fb1ed5-03f7-4f67-b04b-7b265beafee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:56.678340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:56.749098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:56.897871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904df280-f766-4566-8444-2552ed870f6a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:56.899435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-904df280-f766-4566-8444-2552ed870f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:57.045634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904df280-f766-4566-8444-2552ed870f6a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:57.088290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:57.163270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18150105-d915-4a09-961b-5306fe51813f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:57.163796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150105-d915-4a09-961b-5306fe51813f service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:57.164069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150105-d915-4a09-961b-5306fe51813f service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:57.164336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150105-d915-4a09-961b-5306fe51813f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:57.165942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-18150105-d915-4a09-961b-5306fe51813f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:57.254084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18150105-d915-4a09-961b-5306fe51813f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:57.255727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18150105-d915-4a09-961b-5306fe51813f service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:57.274073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 707/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:46:58.406944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fa876-767b-45da-8258-044bc757d74f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:58.407465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fa876-767b-45da-8258-044bc757d74f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:58.407465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fa876-767b-45da-8258-044bc757d74f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:58.408749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:46:58.415063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9b8e-fdfc-4fc9-b4e1-ef7e113c7b2e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:58.415470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9b8e-fdfc-4fc9-b4e1-ef7e113c7b2e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:58.415612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9b8e-fdfc-4fc9-b4e1-ef7e113c7b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:58.470566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:58.493014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-15ff9b8e-fdfc-4fc9-b4e1-ef7e113c7b2e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:46:58.494232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-15ff9b8e-fdfc-4fc9-b4e1-ef7e113c7b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:58.590698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:46:58.741634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fa876-767b-45da-8258-044bc757d74f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:46:58.744184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a79fa876-767b-45da-8258-044bc757d74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:58.889497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fa876-767b-45da-8258-044bc757d74f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:46:58.901400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:46:59.006528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:46:59.007077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:46:59.007360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:46:59.007632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:46:59.009264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:46:59.100501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:46:59.102094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e230c5-3c2a-4d81-9e53-edae8cc60dcb service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:46:59.121774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 714/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:00.225280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:00.225885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:00.225885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:00.227308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:00.234446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc145-9810-4420-8b37-42f21fe4c780 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:00.234925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc145-9810-4420-8b37-42f21fe4c780 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:00.235020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fc145-9810-4420-8b37-42f21fe4c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:00.255827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:00.313060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-798fc145-9810-4420-8b37-42f21fe4c780 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:00.314256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-798fc145-9810-4420-8b37-42f21fe4c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:00.414265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:00.565925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:00.567523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:00.675137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:00.711331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8950a-6a19-49c2-b361-3f686f7c75d0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:00.822439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:00.823077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:00.823353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:00.823620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:00.825222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:00.899127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:00.912564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:00.914163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac14820-fc76-489b-9372-34d7c6e4a234 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:00.932091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 710/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:02.067385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:02.067986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:02.067986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:02.069544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:02.075839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf3509-af6b-491c-9f7c-1044aebad5dd None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:02.076312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf3509-af6b-491c-9f7c-1044aebad5dd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:02.076312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf3509-af6b-491c-9f7c-1044aebad5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:02.151196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fccf3509-af6b-491c-9f7c-1044aebad5dd None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:02.152885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fccf3509-af6b-491c-9f7c-1044aebad5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:02.250920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:02.402921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:02.404521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:02.490607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:02.552151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037466e0-edc2-45d7-bc54-25146443e5bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:02.664856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:02.665378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:02.665652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:02.665924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:02.667548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:02.716168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:02.757336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:02.758950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772eb5c-aee0-4028-ac15-0b52cc0d0a5d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:02.777267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 717/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:03.827732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:03.828152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:03.828284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:03.829646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:03.836038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd76f-ba18-4eb0-ac8a-213cef6ccb40 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:03.836486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd76f-ba18-4eb0-ac8a-213cef6ccb40 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:03.836626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd76f-ba18-4eb0-ac8a-213cef6ccb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:03.911420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-578bd76f-ba18-4eb0-ac8a-213cef6ccb40 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:03.912607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-578bd76f-ba18-4eb0-ac8a-213cef6ccb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:04.008481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:47:04.159595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:04.161139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:04.212177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:04.311319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9b91d-65e1-48ed-9d9c-51e374728146 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:04.423963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:04.424483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:04.424764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:04.425029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:04.426666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:04.433747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:04.518302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:04.519958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aac7b-e9da-4b2d-bf8d-10bf00f1f79b service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:04.537968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 713/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:05.596832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:05.597376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:05.597376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:05.598804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:05.604988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931922-680c-446f-948a-2fe7ce7d84a2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:05.605428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931922-680c-446f-948a-2fe7ce7d84a2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:05.605573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9931922-680c-446f-948a-2fe7ce7d84a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:05.680145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f9931922-680c-446f-948a-2fe7ce7d84a2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:05.681333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f9931922-680c-446f-948a-2fe7ce7d84a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:05.748966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:05.776914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:05.927207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:05.928670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:05.946667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:06.076256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719e406-4507-45ea-ae91-dc735ef3790b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:06.164208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:06.191322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:06.191862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:06.192139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:06.192411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:06.194013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:06.281774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:06.283395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbfb7d-92c9-41d1-9dbb-d2300b7b6f6a service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:06.301823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 720/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:07.372932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:07.373473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:07.373473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:07.374780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:07.380961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d666-657b-4c92-b1ce-45111942ba88 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:07.381419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d666-657b-4c92-b1ce-45111942ba88 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:07.381527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d666-657b-4c92-b1ce-45111942ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:07.459351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bdf5d666-657b-4c92-b1ce-45111942ba88 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:07.460568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bdf5d666-657b-4c92-b1ce-45111942ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:07.502901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:07.557966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:07.708962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:07.710401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:07.857199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91fcb-2fd1-42a0-87c4-2cc784fdb62b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:07.910506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:07.969004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:07.969521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:07.969796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:07.970063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:07.971679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:08.061016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:08.062687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd1e2-cd09-48e8-810d-25429af74c07 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:08.080641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 716/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:09.131690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:09.132182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:09.132294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:09.133580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:09.139925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6b6f-34fc-4e9c-b941-f12d7244d826 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:09.140357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6b6f-34fc-4e9c-b941-f12d7244d826 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:09.140463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6b6f-34fc-4e9c-b941-f12d7244d826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:09.216186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3bdd6b6f-34fc-4e9c-b941-f12d7244d826 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:09.217378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3bdd6b6f-34fc-4e9c-b941-f12d7244d826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:09.219437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:09.313344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:09.464411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:09.466660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:09.611567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235e3bc-81f3-4bb8-a310-dcd1ecd0080e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:09.638539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:09.726608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:09.727173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:09.727489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:09.727767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:09.729368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:09.818704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:09.820568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b490c64-8af4-4af7-974c-6210b5b73c17 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:09.839067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 723/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:11.928606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:11.929187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:11.929187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:11.930558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:11.935922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c02d-56ff-4e7b-84ea-8cf7487521b1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:11.936311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c02d-56ff-4e7b-84ea-8cf7487521b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:11.936475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c02d-56ff-4e7b-84ea-8cf7487521b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:11.977499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:12.003369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7f86c02d-56ff-4e7b-84ea-8cf7487521b1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:12.004497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7f86c02d-56ff-4e7b-84ea-8cf7487521b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:12.218290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:12.533742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:12.535329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:12.639344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba533b-717c-49d6-92fe-ed1a55a46f37 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:12.671946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:12.713213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:12.713805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:12.714080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:12.714349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:12.716487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:12.827432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:12.828804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9dd29-ffc0-407d-b404-d60c411f24a1 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:12.848889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 719/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:15.196607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:15.197158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:15.197158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:15.198498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:15.204409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8207b-937c-47a4-9d86-32fe57a89495 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:15.204861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8207b-937c-47a4-9d86-32fe57a89495 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:15.204970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8207b-937c-47a4-9d86-32fe57a89495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:15.270481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5ee8207b-937c-47a4-9d86-32fe57a89495 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:15.271701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5ee8207b-937c-47a4-9d86-32fe57a89495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:15.284859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:15.367173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:15.654878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:15.656450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:15.759029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e189-f1d8-40c9-97c8-2500243ba2e4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:15.771539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:15.832663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:15.833274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:15.833556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:15.833830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:15.835527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:15.943532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:15.944870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4547733-eb7b-4821-827d-93b31807f0f4 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:15.963257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 726/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:18.495504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:18.496040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:18.496040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:18.498124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:18.503408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a144b4-b116-4053-82d7-7323dd716073 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:18.503815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a144b4-b116-4053-82d7-7323dd716073 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:18.503964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a144b4-b116-4053-82d7-7323dd716073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:18.551270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:18.569942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-56a144b4-b116-4053-82d7-7323dd716073 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:18.571448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-56a144b4-b116-4053-82d7-7323dd716073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:18.667931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:18.955829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:18.957403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:19.060415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26ab0f-3b5d-4938-8ba0-5c0169fd60bf None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:19.101180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:19.136617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:19.137277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:19.137602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:19.137949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:19.139600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:19.249470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:19.250877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf4420-edb2-4967-bdbd-cdf0f9490a57 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:19.268875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 722/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:21.651079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:21.651659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:21.651659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:21.653198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:21.658469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4532e6-4451-4c65-afad-c1780881414b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:21.658904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4532e6-4451-4c65-afad-c1780881414b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:21.659078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4532e6-4451-4c65-afad-c1780881414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:21.725305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8c4532e6-4451-4c65-afad-c1780881414b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:21.726546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8c4532e6-4451-4c65-afad-c1780881414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:21.750583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:21.826739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:22.122543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:22.124081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:22.225758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae093bdd-b62f-49c8-b17f-0a60a90ac586 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:22.246114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:22.299780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:22.300381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:22.300654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:22.300922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:22.302539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:22.410654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:22.412004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34941f8b-3bfd-48f4-9fd4-151927b27921 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:22.430072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 729/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:24.702301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:24.702871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:24.702871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:24.704237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:24.709447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f787a-fc0a-46fb-993b-f6c946d3b8a4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:24.709886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f787a-fc0a-46fb-993b-f6c946d3b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:24.709985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f787a-fc0a-46fb-993b-f6c946d3b8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:24.773699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:24.776639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-298f787a-fc0a-46fb-993b-f6c946d3b8a4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:24.779143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-298f787a-fc0a-46fb-993b-f6c946d3b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:24.876770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:25.162923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:25.164475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:25.269029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930184b-d527-4673-a8a2-684e6bf3fe37 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:25.318526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:25.343645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:25.344247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:25.344521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:25.344788 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:25.346379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:25.454219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:25.455609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11770c9e-6d4f-4f49-ae51-a2bf72ae92d7 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:25.473779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 725/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:27.679059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215278-ce8a-455b-9511-92ee5378617f None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:27.679633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215278-ce8a-455b-9511-92ee5378617f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:27.679633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215278-ce8a-455b-9511-92ee5378617f None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:27.680973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:27.687389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa13e4-5c55-49d0-951d-030111f19c86 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:27.687838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa13e4-5c55-49d0-951d-030111f19c86 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:27.687948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa13e4-5c55-49d0-951d-030111f19c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:27.752939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-54fa13e4-5c55-49d0-951d-030111f19c86 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:27.754129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-54fa13e4-5c55-49d0-951d-030111f19c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:27.782911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:27.853808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:28.136011 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53215278-ce8a-455b-9511-92ee5378617f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:28.137747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-53215278-ce8a-455b-9511-92ee5378617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:28.242687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53215278-ce8a-455b-9511-92ee5378617f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:28.271088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:28.316026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:28.316629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:28.316908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:28.317176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:28.318807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:28.427268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:28.428615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b91709-b8a4-4582-9dd9-aaf8414f72e8 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:28.446611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 732/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:30.651186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:30.651749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:30.651749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:30.653696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:30.658950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59c0c0-afde-4efe-8e7c-6d038ca25324 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:30.659359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59c0c0-afde-4efe-8e7c-6d038ca25324 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:30.659500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59c0c0-afde-4efe-8e7c-6d038ca25324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:30.724990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4e59c0c0-afde-4efe-8e7c-6d038ca25324 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:30.726196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4e59c0c0-afde-4efe-8e7c-6d038ca25324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:30.734751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:30.823777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:31.113251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:31.114841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:31.220492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b4a84-8c8c-4242-a614-a7c0df3a2494 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:31.275636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:31.295518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:31.296157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:31.296444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:31.296724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:31.298401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:31.409889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:31.411910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e540ad5-fc5a-48c1-8d03-c2480bdc0d6d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:31.429931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 728/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:33.631827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785763-7baf-4467-a45d-74718228f21c None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:33.632411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785763-7baf-4467-a45d-74718228f21c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:33.632411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785763-7baf-4467-a45d-74718228f21c None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:33.633734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:33.639151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013936-2263-4569-8c2a-9b734e1ecf3c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:33.639620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013936-2263-4569-8c2a-9b734e1ecf3c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:33.639726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013936-2263-4569-8c2a-9b734e1ecf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:33.705092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-41013936-2263-4569-8c2a-9b734e1ecf3c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:33.706285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-41013936-2263-4569-8c2a-9b734e1ecf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:33.745059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:33.802361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:34.087609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb785763-7baf-4467-a45d-74718228f21c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:34.089198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb785763-7baf-4467-a45d-74718228f21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:34.196045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb785763-7baf-4467-a45d-74718228f21c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:34.238410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:34.277879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:34.278818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:34.279100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:34.279374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:34.281049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:34.395077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:34.396617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbc937-1184-47fe-adb6-86cd3cde14c0 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:34.414855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 735/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:36.607432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:36.608006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:36.608006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:36.609318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:36.614782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443a2f-4dae-474c-bf2a-6c64d438f0d7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:36.615217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443a2f-4dae-474c-bf2a-6c64d438f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:36.615353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443a2f-4dae-474c-bf2a-6c64d438f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:36.680431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-62443a2f-4dae-474c-bf2a-6c64d438f0d7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:36.681561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-62443a2f-4dae-474c-bf2a-6c64d438f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:36.700225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:36.780225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:37.069059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:37.070646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:37.177050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8210-3fc6-4d22-9fc1-a047f92b6703 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:37.244980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:37.252444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:37.253083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:37.253360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:37.253631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:37.255276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:37.362361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:37.363824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57982951-85dd-4b76-802f-064c58eb1aab service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:37.382232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 731/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:39.627770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:39.628321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:39.628321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:39.629665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:47:39.634942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add8f0-b2e2-4e65-9cfc-2b9f5d5fd596 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:39.635407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add8f0-b2e2-4e65-9cfc-2b9f5d5fd596 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:39.635521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add8f0-b2e2-4e65-9cfc-2b9f5d5fd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:39.703376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a9add8f0-b2e2-4e65-9cfc-2b9f5d5fd596 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:47:39.704572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a9add8f0-b2e2-4e65-9cfc-2b9f5d5fd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:39.752980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:39.802415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:47:40.092405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:47:40.093968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:40.197012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351db8f-59f2-4adb-8595-6a14c49610ef None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:47:40.246463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:40.272535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:47:40.273144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:47:40.273422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:47:40.273693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:47:40.275323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:47:40.383960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:47:40.385378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbaf3f-1889-4dc9-900f-41dc856f0be7 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:47:40.403170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 738/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:47:49.800912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=22 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:47:50.400918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=66 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:00.408240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f3089-01c1-4c05-bcd9-a19dd3aca250 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:00.408780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f3089-01c1-4c05-bcd9-a19dd3aca250 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:00.408780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f3089-01c1-4c05-bcd9-a19dd3aca250 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:00.410060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 733/1471] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:00.415453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433d6d-3321-48da-9125-1cb7969d8910 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:00.415867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433d6d-3321-48da-9125-1cb7969d8910 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:00.416006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66433d6d-3321-48da-9125-1cb7969d8910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:00.483612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-66433d6d-3321-48da-9125-1cb7969d8910 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:00.484802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-66433d6d-3321-48da-9125-1cb7969d8910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:00.579248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 739/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:48:00.878146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:00.878434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:00.878791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:00.880156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 734/1473] 10.222.0.15 () {54 vars in 966 bytes} [Mon Sep 12 11:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:00.884980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c5f3-1462-4393-bc0f-a8a9420f8a17 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:00.885269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c5f3-1462-4393-bc0f-a8a9420f8a17 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:00.885548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c5f3-1462-4393-bc0f-a8a9420f8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:00.961838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-02a4c5f3-1462-4393-bc0f-a8a9420f8a17 None None] MFA Rules not processed for user `02a59e7ff79249eaa7f45cdbe15d0eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:00.963080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-02a4c5f3-1462-4393-bc0f-a8a9420f8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:00.983447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:01.039645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 740/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Sep 12 11:48:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:01.047447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:01.049012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:01.159643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd309-3fc0-4dcb-be28-79c984cf1ce2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:01.235773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:01.236376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:01.236653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:01.236921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:01.238533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:01.291440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:01.349975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:01.351457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cbf22-863e-47f7-91f9-813f429fcdcf service glance] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:01.370129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 735/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:03.603184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:03.603768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:03.603768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:03.605061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 741/1476] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:03.610315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28346779-39b3-46d3-b74f-81c63725fc99 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:03.610780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28346779-39b3-46d3-b74f-81c63725fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:03.610947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28346779-39b3-46d3-b74f-81c63725fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:03.676264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-28346779-39b3-46d3-b74f-81c63725fc99 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:03.677460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-28346779-39b3-46d3-b74f-81c63725fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:03.772797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 736/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:04.048141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:04.049675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:04.186355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea062a-9a47-494a-87e2-dc102312f8e4 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:04.238640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:04.283102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:04.283705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:04.283978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:04.284242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:04.284871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:04.286622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:48:04.286622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:48:04.286622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:48:04.288076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd533b-fdee-4e4d-ac04-90b17304d6b2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:04.307304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 742/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Sep 12 11:48:04 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:48:06.541825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:06.542352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:06.542352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:06.544186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 737/1479] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:06.549717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c350fb-0420-4565-973f-d43f7d24f9ad None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:06.550142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c350fb-0420-4565-973f-d43f7d24f9ad None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:06.550259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c350fb-0420-4565-973f-d43f7d24f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:06.617386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e8c350fb-0420-4565-973f-d43f7d24f9ad None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:06.618623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e8c350fb-0420-4565-973f-d43f7d24f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:06.714067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 743/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:07.008730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:07.010321 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:07.020383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:07.160878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e873b-7194-4176-8d9e-dba7d251ef8b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:07.242359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:07.276118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:07.276656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:07.276933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:07.277200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:07.279424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:07.367955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:07.369566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88aef-6a41-4334-8877-6578fad1e503 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:07.387131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 738/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:09.695971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:09.696539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:09.696539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:09.698175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 744/1482] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:09.704120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d206-3c4d-4ee3-95b1-6620a88bef03 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:09.704553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d206-3c4d-4ee3-95b1-6620a88bef03 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:09.704664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d206-3c4d-4ee3-95b1-6620a88bef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:09.768991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0a47d206-3c4d-4ee3-95b1-6620a88bef03 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:09.770175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0a47d206-3c4d-4ee3-95b1-6620a88bef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:09.815815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:09.867171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 739/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:48:10.163326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:10.164894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:10.188981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:10.270318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c4e34-226b-4b4c-aa60-21ae25a9a6b1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:10.341605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:10.342203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:10.342474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:10.342769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:10.344355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:10.420781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:10.456081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:10.457501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63911b-a236-469a-9fd0-d40e20fdb3fd service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:10.475196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 745/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:12.668707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:12.669279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:12.669279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:12.670619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 740/1485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:12.675834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971fd7-fc03-413c-b182-9f80cce228ef None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:12.676234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971fd7-fc03-413c-b182-9f80cce228ef None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:12.676382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971fd7-fc03-413c-b182-9f80cce228ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:12.742605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-22971fd7-fc03-413c-b182-9f80cce228ef None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:12.743804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-22971fd7-fc03-413c-b182-9f80cce228ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:12.838236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 746/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:13.132704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:13.134316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:13.241228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3eb44-7f33-40f6-ad35-b715bb3960d6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:13.316957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:13.318613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:13.319247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:13.319777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:13.320284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:13.322435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:13.433371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:13.435082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb82838-ff4a-40c0-af4c-07609cb94b9d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:13.453684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 741/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:15.764436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:15.765014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:15.765014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:15.766405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 747/1488] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:15.771587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a592e-20d0-45d6-a0bb-063054026f7a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:15.772021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a592e-20d0-45d6-a0bb-063054026f7a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:15.772139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a592e-20d0-45d6-a0bb-063054026f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:15.838069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d01a592e-20d0-45d6-a0bb-063054026f7a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:15.839284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d01a592e-20d0-45d6-a0bb-063054026f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:15.891991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:15.936729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 742/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:16.229012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:16.230641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:16.267675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:16.336131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49e18-0ef7-4cf8-b24b-926465e19937 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:16.409922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:16.410443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:16.410770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:16.411037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:16.412638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:16.492125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:16.521899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:16.523352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd78c76-31d4-4c29-bcdc-31a40d758ff5 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:16.540904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 748/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:18.894400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:18.894991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:18.894991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:18.896312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 743/1491] 10.222.0.15 () {54 vars in 927 bytes} [Mon Sep 12 11:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:48:18.901713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e732-c8f9-4e93-9a8f-56e603c86d3a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:18.902140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e732-c8f9-4e93-9a8f-56e603c86d3a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:18.902278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e732-c8f9-4e93-9a8f-56e603c86d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:18.969338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4916e732-c8f9-4e93-9a8f-56e603c86d3a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:48:18.970579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4916e732-c8f9-4e93-9a8f-56e603c86d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:19.066918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 749/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Sep 12 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:48:19.360533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:48:19.362070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:19.465866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647bac5-ade6-41f9-9376-28788ec3ad89 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:48:19.540219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:48:19.540830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:48:19.541109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:48:19.541381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:48:19.543032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:48:19.552163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:19.773771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:48:19.775163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b2468-bba7-46f5-b223-33726e262fb4 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:48:19.794226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 744/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:48:29.065337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=82 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:48:29.791342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=50 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:32.396787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:32.397393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:32.397393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:32.474347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:49:32.475514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:32.572184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 750/1494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:49:32.580440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:32.580950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:32.581054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:32.655425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:49:32.656617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:32.750590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 745/1495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:49:32.755241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:32.756490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:32.768428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3da7c-df0d-4bda-8ff5-e8045609e5f5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:32.829023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:32.863576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:32.864183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:32.864458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:32.864727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:32.865365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:49:32.865666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:32.866685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:32.866685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:49:32.866685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:49:32.867572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:32.879202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 751/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:49:32.887493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:32.896545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e0f09-1c76-44a6-b6cb-8a964e28d24c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:32.957693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:32.990407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:32.991031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:32.991315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:32.991588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:32.992214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:32.993944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:32.993944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:49:32.993944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:49:32.995187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:33.052661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 746/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:49:32 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:49:33.056918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:33.058445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:33.093287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbc56a-c4dc-4848-996b-4b91b93c5460 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:33.186757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:33.187365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.187643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.187913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.188536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:33.189527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:33.189527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:49:33.189527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:49:33.190758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:33.232961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 752/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:49:33 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:49:33.237027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:33.238629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:33.302030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ae759-5879-495b-9d27-148f83985a6b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:33.384413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:33.395994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:33.396602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.396879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.397152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.397795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:33.398853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:33.398853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:49:33.398853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:49:33.399742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:33.412029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 747/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:49:33.416156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:33.417693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:33.429750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767821-85f7-4eec-8610-1a0093d0774f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:33.455751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:33.527172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:33.527806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.528085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.528361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.529003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:33.530093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:33.530093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:49:33.530093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:49:33.531014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f186eb-aa29-4c30-aa78-a321dfe70133 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:33.543168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 753/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:49:33.547772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:33.549289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:33.582684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afeb6a-679e-46c4-8104-888051299465 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:33.675797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:33.676484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.676950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.677421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] PATH_INFO: `/v3/projects/1dddaf4544a74ed0bb7113dca41afaf1/users/54ff844c404b45239df52f1c18c0700b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.703263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dddaf4544a74ed0bb7113dca41afaf1, user_id=54ff844c404b45239df52f1c18c0700b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:33.704304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:33.704304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:49:33.704304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:49:33.706227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706ab34-b51c-4bd7-a114-6905799542af admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:33.730311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 748/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:49:33 2022] PUT /identity/v3/projects/1dddaf4544a74ed0bb7113dca41afaf1/users/54ff844c404b45239df52f1c18c0700b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:49:33.743774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387e3a-92fc-4b33-8d6c-cf6cd80ab62e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.744056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387e3a-92fc-4b33-8d6c-cf6cd80ab62e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.744329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387e3a-92fc-4b33-8d6c-cf6cd80ab62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.801875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fc387e3a-92fc-4b33-8d6c-cf6cd80ab62e None None] MFA Rules not processed for user `54ff844c404b45239df52f1c18c0700b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:49:33.803095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fc387e3a-92fc-4b33-8d6c-cf6cd80ab62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:33.882055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 754/1502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:49:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:49:33.895465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.895752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.896030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.897361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 749/1503] 10.222.0.15 () {54 vars in 964 bytes} [Mon Sep 12 11:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:49:33.902121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd9b-1c3b-4f1e-8097-74c714f20682 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:33.902409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd9b-1c3b-4f1e-8097-74c714f20682 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:33.902720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd9b-1c3b-4f1e-8097-74c714f20682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:33.934725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:33.980156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9240bd9b-1c3b-4f1e-8097-74c714f20682 None None] MFA Rules not processed for user `6d11c4cfca65434c9108393d11c1f7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:49:33.981373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9240bd9b-1c3b-4f1e-8097-74c714f20682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:34.056813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 755/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Sep 12 11:49:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:49:34.065266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:34.066829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:34.120995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:34.176042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59c774-039c-4000-98a6-862c220a2a8a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:34.252394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:34.252998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:34.253272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:34.253534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:34.255150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:34.353622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:34.355107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:34.358709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:34.373912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 750/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:49:34.420141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c709-f475-42e5-a8d4-0cf465697aa0 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:34.420579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c709-f475-42e5-a8d4-0cf465697aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:34.420687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c709-f475-42e5-a8d4-0cf465697aa0 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:34.421994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 756/1506] 10.222.0.15 () {52 vars in 830 bytes} [Mon Sep 12 11:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:49:34.425545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:34.426871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:34.525182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fac73-3996-45cd-9e36-cd2f1585d443 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:34.616177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:34.616877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:34.617089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:34.617354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] PATH_INFO: `/v3/users/54ff844c404b45239df52f1c18c0700b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:34.630783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ff844c404b45239df52f1c18c0700b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:34.631809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:34.631809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:49:34.631809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:49:34.633014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:34.643281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:34.699383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ff844c404b45239df52f1c18c0700b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:34.722648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ff844c404b45239df52f1c18c0700b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:34.731675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ff844c404b45239df52f1c18c0700b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:34.746467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ff844c404b45239df52f1c18c0700b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:34.746939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ff844c404b45239df52f1c18c0700b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:34.753395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 751/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:49:34 2022] DELETE /identity/v3/users/54ff844c404b45239df52f1c18c0700b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:49:34.757705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:34.757988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:34.758267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:34.815257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:49:34.816368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:34.890534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:34.902907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 757/1508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:49:34.913462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:34.914755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:35.018316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40894de-2b3a-4e97-be91-820330f2cc1f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:35.081381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:35.101745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:35.102352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:35.102656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:35.102928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:35.104509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:35.212899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:35.214338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655d815-8b34-459c-a0ad-9acff1871d31 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:35.233036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 752/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:49:35.579596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:49:35.580943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:49:35.592699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e055df-4e39-4dbe-aa9b-bb41d49f32f6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:49:35.693334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:49:35.693966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:49:35.694242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:49:35.694529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] PATH_INFO: `/v3/projects/1dddaf4544a74ed0bb7113dca41afaf1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:49:35.705378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dddaf4544a74ed0bb7113dca41afaf1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:49:35.706406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:49:35.706406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:49:35.706406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:49:35.707675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:49:35.765981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dddaf4544a74ed0bb7113dca41afaf1', 'request_id': 'req-fba0aba2-2959-4152-b952-c24473eecfa5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:35.772061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dddaf4544a74ed0bb7113dca41afaf1', 'request_id': 'req-fba0aba2-2959-4152-b952-c24473eecfa5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:35.785420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dddaf4544a74ed0bb7113dca41afaf1', 'request_id': 'req-fba0aba2-2959-4152-b952-c24473eecfa5'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:35.813677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] The token cache is being invalidate because project 1dddaf4544a74ed0bb7113dca41afaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:49:35.813961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:35.814389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fba0aba2-2959-4152-b952-c24473eecfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:49:35.815826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 758/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:49:35 2022] DELETE /identity/v3/projects/1dddaf4544a74ed0bb7113dca41afaf1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:49:45.230338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=62 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:49:45.811793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:51:03.192957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:03.193527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:03.193527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:03.194857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:03.199864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739194c9-25c6-4308-ae2a-90213a50622f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:03.200282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739194c9-25c6-4308-ae2a-90213a50622f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:03.200413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739194c9-25c6-4308-ae2a-90213a50622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:03.284014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-739194c9-25c6-4308-ae2a-90213a50622f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:03.285221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-739194c9-25c6-4308-ae2a-90213a50622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:03.389644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:03.543225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:03.545997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:03.669094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec559e-f0bb-4c4e-9d85-2f67fe446063 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:03.675259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:03.675868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:03.676180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:03.676457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:03.684570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:03.686882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e7d6-ff6d-4635-8883-2157c9422c64 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:03.696169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 754/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:04.782077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad5eb-d8db-458b-8c3e-5c04b356ad57 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:04.782677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad5eb-d8db-458b-8c3e-5c04b356ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:04.782800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dad5eb-d8db-458b-8c3e-5c04b356ad57 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:04.784110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:04.790249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9274c1c-4e42-49ed-8753-4d51faa968b7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:04.790713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9274c1c-4e42-49ed-8753-4d51faa968b7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:04.790860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9274c1c-4e42-49ed-8753-4d51faa968b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:04.810884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a9274c1c-4e42-49ed-8753-4d51faa968b7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:04.812079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a9274c1c-4e42-49ed-8753-4d51faa968b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:04.819922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:04.972317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dad5eb-d8db-458b-8c3e-5c04b356ad57 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:04.987096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dad5eb-d8db-458b-8c3e-5c04b356ad57 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:04.992725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:04.993330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:04.993619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:04.993885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:05.002217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:05.003829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75cd2a-43ac-4ba5-b446-5cd7f226ddb9 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:05.013315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 761/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:05.072398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e793-4e6b-4d26-8115-9cab1acaf899 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:05.072685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e793-4e6b-4d26-8115-9cab1acaf899 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:05.072965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5e793-4e6b-4d26-8115-9cab1acaf899 None None] PATH_INFO: `/v3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:05.074579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 756/1517] 10.222.0.50 () {54 vars in 925 bytes} [Mon Sep 12 11:51:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:51:05.081418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c882a-c21d-498a-a8ca-12ed44ad3b0c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:05.082141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c882a-c21d-498a-a8ca-12ed44ad3b0c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:05.082416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c882a-c21d-498a-a8ca-12ed44ad3b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:05.153695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-751c882a-c21d-498a-a8ca-12ed44ad3b0c None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:05.155636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-751c882a-c21d-498a-a8ca-12ed44ad3b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:05.172287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:51:05.229883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 762/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 12 11:51:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:14.033666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d07fa-1576-419c-ab83-69901c2e8b80 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:14.034098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d07fa-1576-419c-ab83-69901c2e8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:14.034214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d07fa-1576-419c-ab83-69901c2e8b80 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:14.035562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 757/1519] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 12 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:14.040352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b753-b721-4380-8b6c-a201cdde4a39 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:14.040759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b753-b721-4380-8b6c-a201cdde4a39 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:14.040912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b753-b721-4380-8b6c-a201cdde4a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:14.062075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e780b753-b721-4380-8b6c-a201cdde4a39 None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:14.063313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e780b753-b721-4380-8b6c-a201cdde4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:14.071881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 763/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 12 11:51:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:14.080439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d07fa-1576-419c-ab83-69901c2e8b80 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:14.094452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d07fa-1576-419c-ab83-69901c2e8b80 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:14.099382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:14.099978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:14.100259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:14.100530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:14.108749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:14.110345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df4c7e-8b02-4964-8cbf-9e2dfdee7a48 service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:14.120092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 758/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:16.492126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c6c9-985f-45e3-9702-1b130dd7cffd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:16.492671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c6c9-985f-45e3-9702-1b130dd7cffd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:16.492671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c6c9-985f-45e3-9702-1b130dd7cffd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:16.494014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:16.500784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92148200-956d-4152-9cec-72a7654757e1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:16.501435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92148200-956d-4152-9cec-72a7654757e1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:16.501435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92148200-956d-4152-9cec-72a7654757e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:16.521315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-92148200-956d-4152-9cec-72a7654757e1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:16.522585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-92148200-956d-4152-9cec-72a7654757e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:16.530470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:16.684220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8c6c9-985f-45e3-9702-1b130dd7cffd None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:16.699361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8c6c9-985f-45e3-9702-1b130dd7cffd None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:16.705362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:16.705957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:16.706237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:16.706537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:16.714369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:16.715978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66faa-10be-42f2-8cb3-965558aee3ba service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:16.725373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 765/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:51:17.787997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d070-3b22-4c82-8608-e6d98121e907 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:17.788540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d070-3b22-4c82-8608-e6d98121e907 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:17.788540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960d070-3b22-4c82-8608-e6d98121e907 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:17.789851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 11:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:17.796534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19874f-0a6a-4669-aeeb-d4081305a0e4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:17.796969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19874f-0a6a-4669-aeeb-d4081305a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:17.797095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19874f-0a6a-4669-aeeb-d4081305a0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:17.816941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0c19874f-0a6a-4669-aeeb-d4081305a0e4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:17.818134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0c19874f-0a6a-4669-aeeb-d4081305a0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:17.826231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 11:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:51:17.976031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960d070-3b22-4c82-8608-e6d98121e907 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:17.990328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960d070-3b22-4c82-8608-e6d98121e907 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:17.995674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:17.996264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:17.996546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:17.996820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:18.005353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:18.006969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e32c1-479f-4813-b58d-5e39a102c50b service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:18.016706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 761/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:22.020109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:22.020631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:22.020631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:22.022049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:22.027499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054710-36a9-4c01-b9f8-f9acdae7067f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:22.027902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054710-36a9-4c01-b9f8-f9acdae7067f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:22.028058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054710-36a9-4c01-b9f8-f9acdae7067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:22.049380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f8054710-36a9-4c01-b9f8-f9acdae7067f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:22.050641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f8054710-36a9-4c01-b9f8-f9acdae7067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:22.058448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:22.367978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:22.369685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:22.463077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee3701-12e3-4f2c-bf32-99b5a4f9b550 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:22.467509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:22.468117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:22.468399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:22.468674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:22.477588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:22.479042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf612bb-017c-4605-a2ba-a0657dc9687b service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:22.489072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 768/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:24.636495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936375-e0a2-4da4-93fb-06d4112a9b7c None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:24.637049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936375-e0a2-4da4-93fb-06d4112a9b7c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:24.637049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936375-e0a2-4da4-93fb-06d4112a9b7c None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:24.638431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:24.643825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760dfea-d3ce-4d6e-94c0-e3a5d2ca8920 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:24.644285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760dfea-d3ce-4d6e-94c0-e3a5d2ca8920 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:24.644396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760dfea-d3ce-4d6e-94c0-e3a5d2ca8920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:24.664088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9760dfea-d3ce-4d6e-94c0-e3a5d2ca8920 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:24.665282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9760dfea-d3ce-4d6e-94c0-e3a5d2ca8920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:24.674322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:24.961978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16936375-e0a2-4da4-93fb-06d4112a9b7c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:24.973517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16936375-e0a2-4da4-93fb-06d4112a9b7c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:24.977179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:24.977777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:24.978055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:24.978323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:24.988212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:24.989678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e425a-98a2-43e2-b241-ad0ead7f9883 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:24.999189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 764/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:28.672581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:28.673107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:28.673107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:28.674471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:28.679563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf45db1-1009-44d7-88c7-a394fde90756 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:28.679993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf45db1-1009-44d7-88c7-a394fde90756 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:28.680101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf45db1-1009-44d7-88c7-a394fde90756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:28.688065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=81 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:51:28.701281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7bf45db1-1009-44d7-88c7-a394fde90756 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:28.702462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7bf45db1-1009-44d7-88c7-a394fde90756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:28.710143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:29.015032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:29.016745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:29.084204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:51:29.101794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc180-69f6-45a4-9e7c-9e4a0f3061ee None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:29.105418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:29.106022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:29.106327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:29.106717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:29.116093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:29.117520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdc27b-dd5d-4c9e-9dc8-efb260c57eb9 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:29.127083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 771/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:31.265163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:31.265759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:31.265759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:31.267078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 11:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:51:31.272379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae376-f079-447b-bd23-9616528412e4 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:31.272796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae376-f079-447b-bd23-9616528412e4 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:31.272913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae376-f079-447b-bd23-9616528412e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:31.292417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-92eae376-f079-447b-bd23-9616528412e4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:51:31.293600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-92eae376-f079-447b-bd23-9616528412e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:31.301839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 11:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:51:31.608523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:51:31.610299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:51:31.623551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8e527-2083-45b9-8751-273ce6f7d456 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:51:31.626993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:51:31.627578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:51:31.627856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:51:31.628122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:51:31.636902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:51:31.639116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a582a1-8016-4228-9915-0edde72e8781 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:51:31.648265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 767/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:51:41.287517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=11 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:51:41.644215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:55.448465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.449718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.449718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.475323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.476528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.487808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 773/1540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:55.496482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.497059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.497059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.527933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.529162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.543281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 768/1541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:55.547891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.561043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8ba50-609a-4f48-9adb-c6231b55231f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.566152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.566825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.567104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.567375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.568036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:55.568336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.569964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c4bab-c2ad-4bac-8a3a-a4e207f2c551 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:55.588043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 774/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:55.596312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.608549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4870f-2c16-4f1d-b423-262aa997a1e1 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.617515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.617872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.618077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.618452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.619607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.620126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.620613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.621501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.623770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e87b2-b442-4a03-93cb-00af30f14444 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:55.657800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.657800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.664870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 775/1543] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:55.681252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 769/1544] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/projects => generated 391 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:55.685886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.687811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9700c1-5ae3-41c6-8953-af962640a237 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.688331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9700c1-5ae3-41c6-8953-af962640a237 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.688944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9700c1-5ae3-41c6-8953-af962640a237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.705507 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fd22d-5997-439f-b9ff-a04656366d13 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.723668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8d9700c1-5ae3-41c6-8953-af962640a237 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.725253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8d9700c1-5ae3-41c6-8953-af962640a237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.809975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.810818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.811099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.811372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.812140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.814079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:55.827995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 770/1545] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:52:55.833405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.834092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.834622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.870544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 776/1546] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:55.871569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.873197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.878546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.892545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2e667-4dea-49d1-bf9b-2efda6586166 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.897178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 771/1547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:52:55.898422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.903871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.904509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.904789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.905061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.910534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:55.910534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.910534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3e203-d8b6-4d3e-af20-3295abd66cf0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:55.914729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e4021-4641-4237-88eb-c298134ef191 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.928244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.929281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.929809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.930297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.931168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.933270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:55.934176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 777/1548] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:55.943446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cab0-d1c5-4891-87d8-380460aab07c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.943790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cab0-d1c5-4891-87d8-380460aab07c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.944004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482cab0-d1c5-4891-87d8-380460aab07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.951044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 772/1549] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:55.952479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:55.970405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b4460-d369-4e35-b59b-abedeaf76a61 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:55.983797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3482cab0-d1c5-4891-87d8-380460aab07c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:55.984596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:55.985040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3482cab0-d1c5-4891-87d8-380460aab07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:55.985828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:55.986331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:55.986828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:55.987787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:55.990025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.002819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 778/1550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.002819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.002819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.002819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.045712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.046952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.048065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 773/1551] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.049658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.138871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 779/1552] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.140039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.145441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62f3ca-2244-4a2e-af81-88835e4bc10d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.151475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.152058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.152766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.153592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.153861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.155507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f481-3e2f-43a1-91d1-325df50cf169 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.157053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2021eee-7f9d-4cca-9d80-95755d5c217c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.161774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.162399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.162695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.162968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.163616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.163920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.165505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.170044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 774/1553] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:56.171446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041a0c-5a39-487b-a0b0-f7c4270a3587 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.171739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041a0c-5a39-487b-a0b0-f7c4270a3587 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.172018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041a0c-5a39-487b-a0b0-f7c4270a3587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.178465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 780/1554] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.179305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.197844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee092b-4844-4d1a-a5b3-b8a540b2f9dc admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.202624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.203252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.203437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-03041a0c-5a39-487b-a0b0-f7c4270a3587 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.203559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.203788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.204416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.204653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-03041a0c-5a39-487b-a0b0-f7c4270a3587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.206308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72423409-f08f-4f56-b95b-8af9a8dbcc0c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.212806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 775/1555] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.214526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782082-32e5-4a1d-8be2-2064195b6566 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.214837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782082-32e5-4a1d-8be2-2064195b6566 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.215114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782082-32e5-4a1d-8be2-2064195b6566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.234078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 781/1556] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.234078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.234078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.234078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.237654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7b782082-32e5-4a1d-8be2-2064195b6566 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.238912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7b782082-32e5-4a1d-8be2-2064195b6566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.247818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 776/1557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.249251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a0a6-74a2-40d1-a2ae-4908eca5fe46 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.249597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a0a6-74a2-40d1-a2ae-4908eca5fe46 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.249932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a0a6-74a2-40d1-a2ae-4908eca5fe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.252971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.254159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.262242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 782/1558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.263050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.267014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:56.269687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0340a0a6-74a2-40d1-a2ae-4908eca5fe46 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.271805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0340a0a6-74a2-40d1-a2ae-4908eca5fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.272592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcfaaa-9e6c-4e05-90ce-6339aaaeaba6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.277319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.277976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.278257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.278560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] PATH_INFO: `/v3/projects/14feb3620a104bc991453536505ba9e0/users/4b746be623d647e2b8438e79ac779912/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.280904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:56.281611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 777/1559] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.283782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9e4c-055a-487d-ab92-8cc83e3ab9d1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.284082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9e4c-055a-487d-ab92-8cc83e3ab9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.284313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9e4c-055a-487d-ab92-8cc83e3ab9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.298482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14feb3620a104bc991453536505ba9e0, user_id=4b746be623d647e2b8438e79ac779912, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.299565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:52:56.299565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:52:56.299565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:52:56.301577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.305057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e69e9e4c-055a-487d-ab92-8cc83e3ab9d1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.306257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e69e9e4c-055a-487d-ab92-8cc83e3ab9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.315111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 783/1560] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:56 2022] PUT /identity/v3/projects/14feb3620a104bc991453536505ba9e0/users/4b746be623d647e2b8438e79ac779912/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:56.323504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 778/1561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.323504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.323504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.323504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.323924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.339999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.341193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.388223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeaafc-b585-4798-8ce7-314aed4b4cf0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.393568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.394189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.394456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.394770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.395436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.396473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:52:56.396473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:52:56.396473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:52:56.397669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a97bf-dc3b-4620-874f-5f7075cb2987 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.402054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 779/1562] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.402923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.412998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0e5a-08a1-4b26-a053-ba9c846ecd76 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.418833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.419422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.419702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.419979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.420628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.422248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0a0b-03d9-49e7-8674-cf69d3d677a6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.429289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 784/1563] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.429289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.429289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.429289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.435316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 780/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:56.436619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0928-a26b-4ac4-8b03-898977320442 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.436911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0928-a26b-4ac4-8b03-898977320442 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.437183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0928-a26b-4ac4-8b03-898977320442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.448801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.450029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.457773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1f5a0928-a26b-4ac4-8b03-898977320442 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.459051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1f5a0928-a26b-4ac4-8b03-898977320442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.511018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 785/1565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.511797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.520861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236721d8-f6d5-42d6-a75f-5c5ab2777080 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.525428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.529384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 781/1566] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.529942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.529942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.529942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.530253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.530353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.530451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.530560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.530665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.530764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e781d4d-1a2b-4efd-9ad2-59d8b42823e1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.541581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 786/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.542741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.543028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.543303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.555228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.557457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.565371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 782/1568] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.566205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.572100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] MFA Rules not processed for user `4b746be623d647e2b8438e79ac779912`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:56.574093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:56.578782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8122de4-c609-44af-b208-0735ef6e6f4e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.583286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.583879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.584155 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.584424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.585065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.585364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.586379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:52:56.586379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:52:56.586379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:52:56.587288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.598997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 783/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.599758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.612312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f47-a58c-4a4d-8ed5-2996c6e0368b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.619462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.620242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.620687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.621137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.622156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.623797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:52:56.623797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:52:56.623797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:52:56.625806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.628632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:56.657296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 784/1570] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/users => generated 294 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.659446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.662764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 787/1571] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:56.662764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.672241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9e3b7-fc38-4be0-a420-8254bafb9c42 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.673615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658427-f872-4cff-8304-baf8326001d6 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.676825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.677422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.677703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.677974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.678206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.678781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.678897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.679086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.679191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.679362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.680003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.680682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.681771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.692416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 788/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.695856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 785/1573] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:56.695856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.696021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.705628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad6966-c15d-4cda-a394-9d02cd1f5b7f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.706983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15430249-6610-4294-91dd-bc26bdd6b0fa admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.710461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.711094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.711376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.711655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.711798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.712235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.712351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.713031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.713475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.714159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.714466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.715093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.716068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.729255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 789/1574] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.730059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.745788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d19ca-c85b-455d-9a45-33e679925751 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.750183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 786/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.750183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.762553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:56.808509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.809257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.809539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.810270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.810444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:52:56.810776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.812387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.813507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d414a4-accb-4431-afea-dbff257a59bf admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.818074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.818708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.818988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.819264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.819889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.821774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.824585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 790/1576] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:52:56.825384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.834593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3908-121d-4a86-95dd-03a931944beb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.840448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.841034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.841309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.841578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.842213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.844054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.855996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 791/1577] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:56.856805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.860142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 787/1578] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.861149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.868400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e9f20-5aa0-418c-97df-18d597fc239e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.924770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.925373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.925654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.925921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.926499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91863baf-d941-4a9d-816c-43f847173ef2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:56.926651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.928528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.931068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:56.931829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:56.932108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:56.932395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] PATH_INFO: `/v3/projects/92efab98bce54b08899f057951b42617/users/66682d5656d3414eaebba666d4e0771f/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:56.947820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92efab98bce54b08899f057951b42617, user_id=66682d5656d3414eaebba666d4e0771f, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:56.951049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:56.957070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 792/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:56.957932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.964116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 788/1580] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:56 2022] PUT /identity/v3/projects/92efab98bce54b08899f057951b42617/users/66682d5656d3414eaebba666d4e0771f/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:56.968305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:56.987837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.021660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9e3f6-9548-443d-a88a-74f945aab8c4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.032961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31f2f-2144-479b-a42c-b181aacd6767 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.037673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.038284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.038597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.038886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.039512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.041423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.068466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 789/1581] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.072834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.080862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.081480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.081802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.082073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.082744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.083744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.084670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.113460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 793/1582] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:56 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.114282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.148299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b0d39-1494-4513-b7b4-17c711b25763 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.177590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897fafe-5ad1-456e-8bd2-0d0034453ada admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.183049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.183669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.183942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.184218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.184861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.186790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.207432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.208156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.208437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.208709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.209343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.211767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.216846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 794/1583] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.217615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.236754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 790/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 282 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.236754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.248802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6000-979c-4db8-a8fa-1ec259c01771 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.268439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.287568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591de6f5-f67e-43a9-9860-966585b8c8a0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.292303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.292908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.293184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.293448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.294097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.295858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.305669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 795/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.307175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.311379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.312110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.312404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.312671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.313306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.315536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743de427-e0db-48b4-87cc-03d101f496eb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.316458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e4c68-aafc-4337-8e54-883d4aea4ff0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.320962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.321536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.321957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.323129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.323749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.325652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.337460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 791/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.341745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.342034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.342312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.351047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 796/1587] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.351047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.361254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274ad2e-b5bd-4ca8-88c5-0ed82b5eb203 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.361438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:57.362585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:57.366048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.366712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.366995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.367302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.367919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.370603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 792/1588] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:57.370969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.371355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.382799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80c824-d809-424d-8d62-f01a6589ca32 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.387759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.388359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.388639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.388909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.389540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.391542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.396236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 797/1589] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.397123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.409318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1276b-ee41-4c9a-b029-306ecd7ad486 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.416258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 793/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.416419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.416533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.416615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.416615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.416615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.416615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.418442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.426364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df6dd9-8277-4934-aa64-74d5e942ce86 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.432110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.433071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.433399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.433676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.434367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.436054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.438293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.446383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 798/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:57.448196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.449601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 794/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.453986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.460814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046409d-35a2-4c6e-9974-c592b0b970b6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.464610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c03c-addf-4033-b8a8-0e6f9005e405 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.465863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.466558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.466844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.467161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] PATH_INFO: `/v3/projects/0356e6690066437a86fc69833ff4cb69/users/6e3c1f7f6f8649a991d5d8a15eb8826b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.469363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.469958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.471100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.471100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.471315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.472840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.482331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0356e6690066437a86fc69833ff4cb69, user_id=6e3c1f7f6f8649a991d5d8a15eb8826b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.483282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 795/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.484083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.487942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.494919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d839b9-9144-487c-9bb4-8dd31ae3eed2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.500042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.500626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.500900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.501168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.501800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.503389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 799/1594] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:57 2022] PUT /identity/v3/projects/0356e6690066437a86fc69833ff4cb69/users/6e3c1f7f6f8649a991d5d8a15eb8826b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:57.503536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.504279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.517223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 796/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.517223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.527327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f1987-3d45-4405-96fd-33f9814929f9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.562608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db535-9907-430e-a010-f9c2c78ecef4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.566164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.566792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.567070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.567338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.585752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.586900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.587209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.588014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.588841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.590448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.600803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 797/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.601919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.612782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91b90e-7fb0-4a44-b8f3-959064d6b70f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.617420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.618012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.618294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.618591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.619669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.621253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.631354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.631662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 798/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.632608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.636002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.642456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74b3c2-3604-4941-85ca-1681dfb6d81d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.645789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 800/1598] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:57.646608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.647301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.647968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.648273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.648526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.649164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.651747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.656447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfc7b-88f2-43b0-bb2e-3193bb52aa30 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.661847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 799/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.662688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.662837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.663702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.664144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.664422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.665138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.667682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a42c-2ebe-477f-af84-0c9df958c413 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.678337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19bd35-e7ea-4a26-a9cc-c51716efcdd7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.682038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 801/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.683285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.683640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.683984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.687058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.687728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.688021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.688297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.688952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.690676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.691159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.704377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 800/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.705205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.705663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:57.706901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:57.725729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 802/1602] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:57.725729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.727066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f514-ec64-4fd0-bf86-c89e9d0ca91d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.736500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.737192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.737477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.737754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] PATH_INFO: `/v3/projects/a06cc2bf74564d789428c1a2c96d789b/users/29056329551c42bd8a42d752f40de3f3/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.745614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9b2d0-5f45-4316-bf9e-580d1ebb4b95 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.753288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.754330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.754969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.755469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.756543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.756737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06cc2bf74564d789428c1a2c96d789b, user_id=29056329551c42bd8a42d752f40de3f3, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.758351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.760286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.772217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 803/1603] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.773244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.780487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 801/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:57 2022] PUT /identity/v3/projects/a06cc2bf74564d789428c1a2c96d789b/users/29056329551c42bd8a42d752f40de3f3/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:57.781677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.787190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ffd67-5a7f-4bdb-8b34-591557ccc722 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.856121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.856786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.857113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.857424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.858139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.858602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5248-5227-4cf5-b520-29c423921a33 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.859798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.863853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.864594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.864881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.865211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.865899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.867758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.871496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 804/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.875251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.883500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 802/1606] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:57.883500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.890888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6232871-ff55-4806-bcc0-0217d45bf33c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.896259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.896934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.897224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.897498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] PATH_INFO: `/v3/projects/aeedeefa0bd94ad180b19f0d559c9563/users/7c3dbb2ca04745b7bdb8baf40c45a649/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.897736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da86d-8edf-4b84-8034-d351fc4024cb admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:57.904073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:57.904830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:57.905727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:57.905727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] PATH_INFO: `/v3/projects/b68484aca102406a914066e2cc11a452/users/23126c9640d6433e96d393b885f3f829/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:57.917745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeedeefa0bd94ad180b19f0d559c9563, user_id=7c3dbb2ca04745b7bdb8baf40c45a649, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.918008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:57.921048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.929130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68484aca102406a914066e2cc11a452, user_id=23126c9640d6433e96d393b885f3f829, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:57.933454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:57.941234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 805/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:57 2022] PUT /identity/v3/projects/aeedeefa0bd94ad180b19f0d559c9563/users/7c3dbb2ca04745b7bdb8baf40c45a649/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:57.942293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:57.944245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:57.962081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 803/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:57 2022] PUT /identity/v3/projects/b68484aca102406a914066e2cc11a452/users/23126c9640d6433e96d393b885f3f829/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:57.963125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.013238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae07c-9783-453a-9354-3117438540a9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.037218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d315533-2034-42d0-ba6e-0ec7daf3c130 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.043730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.044466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.044756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.045038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] PATH_INFO: `/v3/projects/b58f9aa9cd6e4152a1cd650798aba36e/users/97c8c8b8131746a98c6ceaa0ab2f3994/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.062288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58f9aa9cd6e4152a1cd650798aba36e, user_id=97c8c8b8131746a98c6ceaa0ab2f3994, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.065517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.086688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 804/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:57 2022] PUT /identity/v3/projects/b58f9aa9cd6e4152a1cd650798aba36e/users/97c8c8b8131746a98c6ceaa0ab2f3994/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:58.088852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.106338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.107308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.107594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.108014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.121271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.122500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.133964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 806/1610] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:58.135082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.145200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd920241-f3fc-4558-b9cb-c542a63d7d4f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.160232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:58.183868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ee4c4-a77d-42d9-8bfc-3b98fcdd2cda admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.190507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.191638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.191638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.191875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.192687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.194377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.206963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 805/1611] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:58.207848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.218636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb3e95-2c06-4b39-8999-caa7fc6d85d8 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.219545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.220315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.220660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.221003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] PATH_INFO: `/v3/projects/b8f53720176046919919835156882fff/users/2e316a50d94143809bb686b18d07af8a/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.224951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.225699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.226185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.226555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] PATH_INFO: `/v3/projects/7225740627444e3a97ff730460c52e7d/users/f58a59c389cf4b3198a6c452eb2772d6/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.238324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f53720176046919919835156882fff, user_id=2e316a50d94143809bb686b18d07af8a, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.242278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd452b-2823-45d8-8358-dd1a09f3acf1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.245736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7225740627444e3a97ff730460c52e7d, user_id=f58a59c389cf4b3198a6c452eb2772d6, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.248785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe453-801b-403c-aa94-805da2b5b892 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.270544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 807/1612] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:58 2022] PUT /identity/v3/projects/b8f53720176046919919835156882fff/users/2e316a50d94143809bb686b18d07af8a/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:58.270544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.270544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.270544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.275067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 806/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:58 2022] PUT /identity/v3/projects/7225740627444e3a97ff730460c52e7d/users/f58a59c389cf4b3198a6c452eb2772d6/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:58.278592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258a85-8067-4387-b993-3d32cfad95de None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.278592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258a85-8067-4387-b993-3d32cfad95de None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.278592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258a85-8067-4387-b993-3d32cfad95de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.298060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d3258a85-8067-4387-b993-3d32cfad95de None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.299312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d3258a85-8067-4387-b993-3d32cfad95de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.301319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] MFA Rules not processed for user `7c3dbb2ca04745b7bdb8baf40c45a649`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.302588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.344840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:58.372497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 807/1614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:58.377832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.377832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.377832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.393796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 808/1615] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:58.393796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.400994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd466-e9d2-41db-8998-e8dc29574b24 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.405780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.406436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.406752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.407027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] PATH_INFO: `/v3/projects/a06cc2bf74564d789428c1a2c96d789b/users/29056329551c42bd8a42d752f40de3f3/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.409116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06cc2bf74564d789428c1a2c96d789b, user_id=29056329551c42bd8a42d752f40de3f3, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.411825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55858a2b-eba9-419e-8408-3bd821f25adc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.420963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] MFA Rules not processed for user `97c8c8b8131746a98c6ceaa0ab2f3994`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.422949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.430635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 809/1616] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:58 2022] PUT /identity/v3/projects/a06cc2bf74564d789428c1a2c96d789b/users/29056329551c42bd8a42d752f40de3f3/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:58.433662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d17a6-33ec-438a-84b0-32007f497f0b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.434159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d17a6-33ec-438a-84b0-32007f497f0b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.434797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d17a6-33ec-438a-84b0-32007f497f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.477723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b78d17a6-33ec-438a-84b0-32007f497f0b None None] MFA Rules not processed for user `2e316a50d94143809bb686b18d07af8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.479933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b78d17a6-33ec-438a-84b0-32007f497f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.581182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 810/1617] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:58.581182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.581182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.581182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.594143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 808/1618] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:58.595329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.613390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:58.650279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.651999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.697805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c2462-b6e1-447e-8648-d061a207da8f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.703623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.704411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.704807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.705226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.736848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 811/1619] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:58.737858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:58.804310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.807917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f181672-74e1-4a74-9ae2-27d7838c0091 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.829427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 809/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:58.830404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb17f93-7323-4bef-9eb8-6f8868fa18d2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:58.831684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59464-2ffa-4ffd-a208-0960d779cd87 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.832053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59464-2ffa-4ffd-a208-0960d779cd87 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.832339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59464-2ffa-4ffd-a208-0960d779cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.835553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:58.836585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:58.837061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:58.838169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:58.877458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e7e59464-2ffa-4ffd-a208-0960d779cd87 None None] MFA Rules not processed for user `29056329551c42bd8a42d752f40de3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:58.878745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e7e59464-2ffa-4ffd-a208-0960d779cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:58.889496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:58.964729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:58.966582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:58.991846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 812/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:58.992347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.004836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499845-faff-452f-8b11-3ba49e4910b9 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.008096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 810/1622] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:59.010804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.012017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.012237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea442-1918-4ea5-9732-e6cefd128f16 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.012641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea442-1918-4ea5-9732-e6cefd128f16 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.012791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.013115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea442-1918-4ea5-9732-e6cefd128f16 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.013371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.015123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.017218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.018220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 811/1623] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 12 11:52:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:52:59.027682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ea442-1918-4ea5-9732-e6cefd128f16 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.052956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ea442-1918-4ea5-9732-e6cefd128f16 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.070345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.074453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.074773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.075142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.086601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 813/1624] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:52:58 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:59.088093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.104148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.120385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccffb9a-a74d-4248-9a24-f029613b201f service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.133543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 812/1625] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:59.135341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.135903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.136482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.185218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:59.203741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:52:59.205481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:52:59.220219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833e811-04ff-4faf-b16f-e17a6d18e089 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.226419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.227148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.227433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.227713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.372066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 813/1626] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 12 11:52:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:52:59.373108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.385053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.388476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.419638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 814/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:59.421668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.455872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f521-5141-45ed-9cd4-d6e5dd927751 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.471105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.471761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.472041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.472354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.548725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcb62e-2610-47cb-8974-92f64fd0c4a2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.566884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88695276-4648-4fac-873b-e9638517889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.567634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88695276-4648-4fac-873b-e9638517889b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.567986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88695276-4648-4fac-873b-e9638517889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.568328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88695276-4648-4fac-873b-e9638517889b admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.569481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88695276-4648-4fac-873b-e9638517889b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.571698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88695276-4648-4fac-873b-e9638517889b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.617222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 814/1628] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:52:59 2022] POST /identity/v3/users => generated 309 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:52:59.617943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88695276-4648-4fac-873b-e9638517889b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.627666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.629428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.631615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:52:59.642299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88695276-4648-4fac-873b-e9638517889b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.652837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 815/1629] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:59.653550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.654495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.654862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.655404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.656435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.683764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43691c-ef63-41f6-9017-fea522e9ebe4 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.704973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.705498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.705782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.706055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.706776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.708378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.741188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 816/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:52:59.755265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.784645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0638a-f543-4c72-8007-52e15e1a41ba admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:52:59.786913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.788724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.796940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:52:59.798956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:52:59.799804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:52:59.801279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] PATH_INFO: `/v3/projects/7bc5652c4b824c6986da97734f0775d5/users/eeaa77900e2b4a238a43e751f7939f24/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:52:59.811074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 815/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:52:59.865484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc5652c4b824c6986da97734f0775d5, user_id=eeaa77900e2b4a238a43e751f7939f24, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:52:59.870255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:52:59.917351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 817/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:52:59 2022] PUT /identity/v3/projects/7bc5652c4b824c6986da97734f0775d5/users/eeaa77900e2b4a238a43e751f7939f24/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:52:59.921132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:52:59.935341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d4d-3eb5-400c-bd2e-c93febe4c6d4 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:00.120126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:00.126404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.126646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.126925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.129997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:00.132566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a4ea1-3a25-430f-88e5-5dc52ee093d6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:00.169575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 816/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:00.171187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:00.183422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7430e9-9e1f-43be-9725-fd2deaaf7241 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.183707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7430e9-9e1f-43be-9725-fd2deaaf7241 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.183976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7430e9-9e1f-43be-9725-fd2deaaf7241 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.185320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 817/1634] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 12 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:00.187899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c924eb-353a-4455-b241-f40a6d13d3bf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:00.195358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.195638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.195911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.202809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:00.203488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.203786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.204039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] PATH_INFO: `/v3/projects/7bc5652c4b824c6986da97734f0775d5/users/eeaa77900e2b4a238a43e751f7939f24/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.208886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc5652c4b824c6986da97734f0775d5, user_id=eeaa77900e2b4a238a43e751f7939f24, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:00.211594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165f7d-56e5-441c-a68c-faf3a3938fb6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:00.233087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:00.237166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 818/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:00 2022] PUT /identity/v3/projects/7bc5652c4b824c6986da97734f0775d5/users/eeaa77900e2b4a238a43e751f7939f24/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:00.238071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:00.247436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.247730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.248010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.323686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] MFA Rules not processed for user `eeaa77900e2b4a238a43e751f7939f24`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:00.324946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:00.373634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 818/1636] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:00.379734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:00.383025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:00.471070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:00.487693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 819/1637] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:00.532692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc86d46-bf02-4cdb-9b0e-19fc4d910e97 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:00.544264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:00.544923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.545198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.545473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.579433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:00.581350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:00.606420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 819/1638] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:00.741527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:00.754828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d968b1-0ce4-4542-9386-12ef2899811c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:00.764602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:00.766031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:00.769223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:00.769709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:00.799688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:00.801180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df3e8e-3cb0-4c8f-9882-c183949b5889 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:00.821141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 820/1639] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:00 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:01.392116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:01.549588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcc67c-89ee-4700-82df-93e88cc68ee1 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:01.562124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:01.563310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:01.563715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:01.565424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] PATH_INFO: `/v3/users/97c8c8b8131746a98c6ceaa0ab2f3994` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:01.569510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c8c8b8131746a98c6ceaa0ab2f3994)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:01.577649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:01.704258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c8c8b8131746a98c6ceaa0ab2f3994'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:01.724774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c8c8b8131746a98c6ceaa0ab2f3994'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:01.743082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c8c8b8131746a98c6ceaa0ab2f3994'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:01.757308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f226-3a43-45de-b24a-8d6650eeca36 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:01.757918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f226-3a43-45de-b24a-8d6650eeca36 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:01.758017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c8c8b8131746a98c6ceaa0ab2f3994'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:01.758110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58f226-3a43-45de-b24a-8d6650eeca36 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:01.760596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 821/1640] 10.222.0.15 () {54 vars in 962 bytes} [Mon Sep 12 11:53:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:01.761478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-443efb90-2099-4c23-8366-1ffe53310e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c8c8b8131746a98c6ceaa0ab2f3994'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:01.774544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0e82-a237-46a7-a1ea-5e8e9c1ebc13 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:01.774883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0e82-a237-46a7-a1ea-5e8e9c1ebc13 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:01.776701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f0e82-a237-46a7-a1ea-5e8e9c1ebc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:01.780276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 820/1641] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:01 2022] DELETE /identity/v3/users/97c8c8b8131746a98c6ceaa0ab2f3994 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:01.785107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:01.785484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:01.785882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:01.830771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:01.832868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:01.837860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-aa3f0e82-a237-46a7-a1ea-5e8e9c1ebc13 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:01.843377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aa3f0e82-a237-46a7-a1ea-5e8e9c1ebc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:01.938947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:01.975550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 821/1642] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:01.998392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:01.999503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:02.015331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 822/1643] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Sep 12 11:53:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:02.019802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947688-b1e9-46dc-8bf9-895907404e03 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:02.020707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947688-b1e9-46dc-8bf9-895907404e03 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:02.022346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21947688-b1e9-46dc-8bf9-895907404e03 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:02.026059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 823/1644] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 12 11:53:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:02.032325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:02.032898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:02.033412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:02.074852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:02.083794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:02.111258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 824/1645] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:53:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:02.125357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:02.144148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:02.149064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb789a5-764f-426d-9247-ac2f15922131 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:02.154557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:02.155197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:02.155475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:02.155746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:02.166562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:02.168091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:02.189273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 822/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:02.348050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36d5f0-e65a-41a7-bc51-ad25361b4923 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:02.394477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:02.394477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:02.394477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:02.394477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:02.409756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:02.411270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:02.450054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 825/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:03.454409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:03.456015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:03.499546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf87989-1662-416a-a5dd-774bbc817ea5 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:03.517407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:03.518074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:03.518354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:03.518661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] PATH_INFO: `/v3/projects/b58f9aa9cd6e4152a1cd650798aba36e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:03.523586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58f9aa9cd6e4152a1cd650798aba36e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:03.524630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: WARNING py.warnings [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:53:03.524630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: warnings.warn(msg) Sep 12 11:53:03.524630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: Sep 12 11:53:03.525857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:03.734916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58f9aa9cd6e4152a1cd650798aba36e', 'request_id': 'req-29fe915d-0255-45e4-8b3b-eba362630712'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:03.750926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58f9aa9cd6e4152a1cd650798aba36e', 'request_id': 'req-29fe915d-0255-45e4-8b3b-eba362630712'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:03.764603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58f9aa9cd6e4152a1cd650798aba36e', 'request_id': 'req-29fe915d-0255-45e4-8b3b-eba362630712'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:03.812198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] The token cache is being invalidate because project b58f9aa9cd6e4152a1cd650798aba36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:03.812535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:03.813303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:03.819257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 823/1648] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:03 2022] DELETE /identity/v3/projects/b58f9aa9cd6e4152a1cd650798aba36e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:03.823488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:03.838247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028bc5-b8f5-4b2c-af12-c7be2e0a5b57 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:03.890984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:03.926844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:03.928194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:03.928886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:03.929361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] PATH_INFO: `/v3/users/eeaa77900e2b4a238a43e751f7939f24` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:03.932972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeaa77900e2b4a238a43e751f7939f24)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:03.938889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: WARNING py.warnings [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 11:53:03.938889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: warnings.warn(msg) Sep 12 11:53:03.938889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: Sep 12 11:53:03.961078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:04.214786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeaa77900e2b4a238a43e751f7939f24'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:04.232306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeaa77900e2b4a238a43e751f7939f24'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:04.282151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeaa77900e2b4a238a43e751f7939f24'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:04.286870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeaa77900e2b4a238a43e751f7939f24'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:04.316993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7eedb198-2088-481e-b419-1c7c44706379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeaa77900e2b4a238a43e751f7939f24'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:04.331457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 826/1649] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:03 2022] DELETE /identity/v3/users/eeaa77900e2b4a238a43e751f7939f24 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:05.991862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:05.994707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:06.020574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe915d-0255-45e4-8b3b-eba362630712 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:06.187486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:06.188083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:06.188363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:06.188835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] PATH_INFO: `/v3/projects/7bc5652c4b824c6986da97734f0775d5` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:06.190413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc5652c4b824c6986da97734f0775d5)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:06.193777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:06.244963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:06.305391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc5652c4b824c6986da97734f0775d5', 'request_id': 'req-44e19847-e50a-4c43-9ac0-235f4a0b0d39'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:06.320648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc5652c4b824c6986da97734f0775d5', 'request_id': 'req-44e19847-e50a-4c43-9ac0-235f4a0b0d39'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:06.345290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc5652c4b824c6986da97734f0775d5', 'request_id': 'req-44e19847-e50a-4c43-9ac0-235f4a0b0d39'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:06.366813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] The token cache is being invalidate because project 7bc5652c4b824c6986da97734f0775d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:06.367120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:06.369348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-44e19847-e50a-4c43-9ac0-235f4a0b0d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:06.374403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 824/1650] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:05 2022] DELETE /identity/v3/projects/7bc5652c4b824c6986da97734f0775d5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:06.378469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:06.378966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:06.379046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:06.427875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:06.429112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:06.632553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 827/1651] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:06.643626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:06.643987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:06.644330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:06.682852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:06.684359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:06.700466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 825/1652] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:06.710973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:06.730874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c065f-3ef6-4d5f-ae70-dc40b150de0f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:06.756508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:06.757155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:06.757436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:06.757723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:06.758383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:06.759656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:06.763377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:06.827062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 828/1653] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:06.828308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:06.854886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e563f-1b7d-42fb-9339-b9ed753451f5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:06.864238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:06.864890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:06.865172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:06.865449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:06.866895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:06.868063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:06.929379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 826/1654] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:06 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:06.930179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:06.951187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78115b2b-df32-49a4-96d6-b53ecb82f762 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:07.066716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:07.111422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:07.112060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.112853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.112853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.113272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:07.118297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:07.168010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 829/1655] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:06 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:07.176269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:07.212719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff425f19-69c4-450e-bbcf-5bc966cdb883 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:07.236402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:07.255275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.258149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.258149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.259965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:07.263893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:07.283281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 827/1656] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:07.293678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:07.311638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4547e6-9be5-4352-b2ca-6d58bc7c35ba admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:07.323596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:07.323799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.324073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.324241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.324920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:07.326475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4978002-4c41-4fdb-ae75-c74294b2cc22 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:07.366693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 830/1657] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:07.371004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:07.393722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431bcc-7a2b-4ccf-a366-4b974b41b8cf admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:07.400455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:07.401148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.401427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.401845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] PATH_INFO: `/v3/projects/f272e057cf534d80a3de83d92c1c5ef7/users/44af54fb43964f15a56e35c336cef0fc/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.420323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f272e057cf534d80a3de83d92c1c5ef7, user_id=44af54fb43964f15a56e35c336cef0fc, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:07.423696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:07.461333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.461995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.461995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.466497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 828/1658] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:07 2022] PUT /identity/v3/projects/f272e057cf534d80a3de83d92c1c5ef7/users/44af54fb43964f15a56e35c336cef0fc/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:07.523474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] MFA Rules not processed for user `44af54fb43964f15a56e35c336cef0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:07.526742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:07.730070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 831/1659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:07.734271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:07.866018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35d597-1bd2-413b-92bb-9e6574835684 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:07.881515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:07.883252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:07.884179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:07.885210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:07.885960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:07.891594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:07.970054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 829/1660] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:07 2022] POST /identity/v3/projects => generated 373 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:07.973873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:07.997623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec5e55-7129-4a2a-a0e4-65ab5f8e267b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.138938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.139585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.139863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.140136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.140960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.143045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.195399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 832/1661] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:07 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:08.196787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:08.211027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dee136-74d4-42b9-a794-5f41070575ae admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.226975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.228208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.228650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.228929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.229734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.233834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.259838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 830/1662] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:08.261058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:08.275838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06503665-3be9-479f-94da-9df12ace455a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.285464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.286211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.286615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.286786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] PATH_INFO: `/v3/projects/03706b8ba61d4da38228e23d677346bd/users/fa21ad0ccfbe40b28a0978dac19a3644/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.323370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03706b8ba61d4da38228e23d677346bd, user_id=fa21ad0ccfbe40b28a0978dac19a3644, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.326908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.350779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 833/1663] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:08 2022] PUT /identity/v3/projects/03706b8ba61d4da38228e23d677346bd/users/fa21ad0ccfbe40b28a0978dac19a3644/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:08.355845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:08.371906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c17c-729c-445a-890d-d8c8187cebfd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.391124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:08.490542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.491355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.491924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.492300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.493429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.495503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803293e5-c210-4c3d-bfd7-e3cba0d2b071 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.513984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 831/1664] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:08.516118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:08.530920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71c6e8-c7e5-4af3-8b15-9a22fbff9f49 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.539860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.541425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.541811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.542338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] PATH_INFO: `/v3/projects/03706b8ba61d4da38228e23d677346bd/users/fa21ad0ccfbe40b28a0978dac19a3644/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.545153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03706b8ba61d4da38228e23d677346bd, user_id=fa21ad0ccfbe40b28a0978dac19a3644, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.549152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.574040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 834/1665] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:08 2022] PUT /identity/v3/projects/03706b8ba61d4da38228e23d677346bd/users/fa21ad0ccfbe40b28a0978dac19a3644/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:08.576090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1899-7a68-4b6a-b9fa-a2e9f43318d1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.576485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1899-7a68-4b6a-b9fa-a2e9f43318d1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.576864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1899-7a68-4b6a-b9fa-a2e9f43318d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.614081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-193c1899-7a68-4b6a-b9fa-a2e9f43318d1 None None] MFA Rules not processed for user `fa21ad0ccfbe40b28a0978dac19a3644`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:08.616150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-193c1899-7a68-4b6a-b9fa-a2e9f43318d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:08.722567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 832/1666] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:08.731114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:08.733080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:08.744038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:08.846346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e9394-7188-429a-83b4-23c69ad71495 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:08.853062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:08.854734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:08.855170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:08.855607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:08.869555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:08.871445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:08.884630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 835/1667] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:09.637847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:09.638394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:09.638727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:09.734173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] MFA Rules not processed for user `6e3c1f7f6f8649a991d5d8a15eb8826b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:09.735495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:09.865094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 833/1668] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:09.879243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:09.881706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:09.937039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:10.019082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:10.064279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfa734-d903-409e-ab39-436d547b3623 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:10.076997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:10.077629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:10.077917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:10.078190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:10.090801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900424e3-191c-40be-9253-aba2d37b3a15 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:10.101881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:10.104550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab8e6-b690-4d67-ba2a-d5e122457e55 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:10.118831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:10.119140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:10.120724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:10.120724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] PATH_INFO: `/v3/users/44af54fb43964f15a56e35c336cef0fc` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:10.121670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44af54fb43964f15a56e35c336cef0fc)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:10.123350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 836/1669] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:10.140853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:10.236882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44af54fb43964f15a56e35c336cef0fc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:10.266940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44af54fb43964f15a56e35c336cef0fc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:10.287108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44af54fb43964f15a56e35c336cef0fc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:10.311454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44af54fb43964f15a56e35c336cef0fc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:10.312771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44af54fb43964f15a56e35c336cef0fc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:10.322869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 834/1670] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:09 2022] DELETE /identity/v3/users/44af54fb43964f15a56e35c336cef0fc => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:10.327160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728233-8f58-4829-850c-1106a258e15f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:10.328661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728233-8f58-4829-850c-1106a258e15f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:10.329054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728233-8f58-4829-850c-1106a258e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:10.370931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bb728233-8f58-4829-850c-1106a258e15f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:10.383051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb728233-8f58-4829-850c-1106a258e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:10.487769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 837/1671] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:10.521197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:10.524681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:10.657380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9300f-fe6d-4436-9188-791aafab944d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:10.665724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:10.666803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:10.674238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:10.674942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:10.693262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:10.694941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:10.717981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 835/1672] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:11.457093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728233-8f58-4829-850c-1106a258e15f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:11.458624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb728233-8f58-4829-850c-1106a258e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:11.476554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728233-8f58-4829-850c-1106a258e15f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:11.487958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:11.488737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:11.489506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:11.490437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] PATH_INFO: `/v3/users/4b746be623d647e2b8438e79ac779912` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:11.492894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b746be623d647e2b8438e79ac779912)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:11.499255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:11.548739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:11.552491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:11.556255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:11.589254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666125-ee22-44c4-b637-5fe7ec42b9e4 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:11.598128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b746be623d647e2b8438e79ac779912'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.601011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:11.608472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:11.609079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:11.609220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] PATH_INFO: `/v3/projects/f272e057cf534d80a3de83d92c1c5ef7` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:11.611343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b746be623d647e2b8438e79ac779912'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.612353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f272e057cf534d80a3de83d92c1c5ef7)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:11.616507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:11.665859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b746be623d647e2b8438e79ac779912'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.666855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b746be623d647e2b8438e79ac779912'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.676121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4b7a81ab-597f-42a7-84a5-16a50ea69bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b746be623d647e2b8438e79ac779912'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.698481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 838/1673] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:11 2022] DELETE /identity/v3/users/4b746be623d647e2b8438e79ac779912 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:11.700242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:11.700580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:11.700912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:11.734583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:11.736474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:11.737783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:11.746461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f272e057cf534d80a3de83d92c1c5ef7', 'request_id': 'req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.763243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f272e057cf534d80a3de83d92c1c5ef7', 'request_id': 'req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.772010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f272e057cf534d80a3de83d92c1c5ef7', 'request_id': 'req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.810457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] The token cache is being invalidate because project f272e057cf534d80a3de83d92c1c5ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:11.810811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.811544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:11.813169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 836/1674] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:11 2022] DELETE /identity/v3/projects/f272e057cf534d80a3de83d92c1c5ef7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:11.817435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:11.823287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:11.903599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 839/1675] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:11.972457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:11.973993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:12.027302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2351-fb59-418b-89e0-b5f10dd1e59e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:12.066982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:12.071234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:12.074017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:12.074936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] PATH_INFO: `/v3/users/fa21ad0ccfbe40b28a0978dac19a3644` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:12.086549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa21ad0ccfbe40b28a0978dac19a3644)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:12.089005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:12.119133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1023661-6aaf-4b66-8eaf-a375d38773ab None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:12.132355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:12.134202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:12.134487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:12.134787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:12.164419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:12.169639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:12.200216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 840/1676] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:12.243805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa21ad0ccfbe40b28a0978dac19a3644'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:12.269275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa21ad0ccfbe40b28a0978dac19a3644'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:12.294653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa21ad0ccfbe40b28a0978dac19a3644'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:12.307396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa21ad0ccfbe40b28a0978dac19a3644'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:12.308717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa21ad0ccfbe40b28a0978dac19a3644'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:12.327320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 837/1677] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:11 2022] DELETE /identity/v3/users/fa21ad0ccfbe40b28a0978dac19a3644 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:13.200410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:13.204510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:13.249324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be993cc-7680-4ea1-a19e-2bd02527fa77 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:13.356243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:13.358725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:13.444567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:13.492812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:13.493629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:13.494189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:13.495116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] PATH_INFO: `/v3/users/2e316a50d94143809bb686b18d07af8a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:13.498604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:13.503122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e316a50d94143809bb686b18d07af8a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:13.506461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:13.536998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636946e6-ce61-491a-b4b9-eed8019a3bf9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:13.552940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:13.557752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:13.558878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:13.561142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] PATH_INFO: `/v3/projects/03706b8ba61d4da38228e23d677346bd` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:13.566769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03706b8ba61d4da38228e23d677346bd)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:13.569732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:13.628837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e316a50d94143809bb686b18d07af8a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.668237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e316a50d94143809bb686b18d07af8a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.713403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03706b8ba61d4da38228e23d677346bd', 'request_id': 'req-03bbf73b-14a0-4128-9b5d-693335f02da6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.729874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03706b8ba61d4da38228e23d677346bd', 'request_id': 'req-03bbf73b-14a0-4128-9b5d-693335f02da6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.730875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e316a50d94143809bb686b18d07af8a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.732168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e316a50d94143809bb686b18d07af8a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.746779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03706b8ba61d4da38228e23d677346bd', 'request_id': 'req-03bbf73b-14a0-4128-9b5d-693335f02da6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.755393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-12d9f8ef-ba34-49c7-986c-0321623c0d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e316a50d94143809bb686b18d07af8a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.781405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] The token cache is being invalidate because project 03706b8ba61d4da38228e23d677346bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:13.782076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.783535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03bbf73b-14a0-4128-9b5d-693335f02da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:13.786432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 838/1678] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:13 2022] DELETE /identity/v3/projects/03706b8ba61d4da38228e23d677346bd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:13.788038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ee06-b4ea-4189-90e3-929016264dfa None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:13.788544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ee06-b4ea-4189-90e3-929016264dfa None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:13.789076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30ee06-b4ea-4189-90e3-929016264dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:13.789458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 841/1679] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:13 2022] DELETE /identity/v3/users/2e316a50d94143809bb686b18d07af8a => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:13.794712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:13.797865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:13.798184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:13.875888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0f30ee06-b4ea-4189-90e3-929016264dfa None None] MFA Rules not processed for user `66682d5656d3414eaebba666d4e0771f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:13.877570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0f30ee06-b4ea-4189-90e3-929016264dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:13.892296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:13.895528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.102676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 839/1680] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.104623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54152b-b558-4b14-8cd6-a61c76b9a861 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.106606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54152b-b558-4b14-8cd6-a61c76b9a861 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.107106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54152b-b558-4b14-8cd6-a61c76b9a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.160853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 842/1681] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.161705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:14.167161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.194230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c84406-0d12-4df8-a7f1-39b816b4b8a8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:14.205790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:14.206454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.206955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.207382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] PATH_INFO: `/v3/projects/14feb3620a104bc991453536505ba9e0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.208448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9e54152b-b558-4b14-8cd6-a61c76b9a861 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:14.209218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14feb3620a104bc991453536505ba9e0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:14.209741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9e54152b-b558-4b14-8cd6-a61c76b9a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.211498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:14.246726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 840/1682] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.248419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.249835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.251069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.338944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14feb3620a104bc991453536505ba9e0', 'request_id': 'req-dbd11909-d056-42e4-b3ea-26ebc5a21d72'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:14.346992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] MFA Rules not processed for user `66682d5656d3414eaebba666d4e0771f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:14.365163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14feb3620a104bc991453536505ba9e0', 'request_id': 'req-dbd11909-d056-42e4-b3ea-26ebc5a21d72'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:14.369460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.375134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14feb3620a104bc991453536505ba9e0', 'request_id': 'req-dbd11909-d056-42e4-b3ea-26ebc5a21d72'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:14.376979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:14.431647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] The token cache is being invalidate because project 14feb3620a104bc991453536505ba9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:14.431947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:14.432629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-dbd11909-d056-42e4-b3ea-26ebc5a21d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:14.435589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 843/1683] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:14 2022] DELETE /identity/v3/projects/14feb3620a104bc991453536505ba9e0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:14.439781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ce53-fa8b-4919-b684-ecb3645dff4b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.440068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ce53-fa8b-4919-b684-ecb3645dff4b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.440347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ce53-fa8b-4919-b684-ecb3645dff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.488089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5f47ce53-fa8b-4919-b684-ecb3645dff4b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:14.489349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5f47ce53-fa8b-4919-b684-ecb3645dff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.561770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 841/1684] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.573322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:14.575498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.615045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:14.676871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 844/1685] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.684270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.686114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.686694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.752029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] MFA Rules not processed for user `23126c9640d6433e96d393b885f3f829`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:14.753801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.771876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6f8c0-cee4-47a2-a8c3-a4791522a286 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:14.778399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:14.779439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.779915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.780592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.787830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.818132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:14.820003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:14.848575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 842/1686] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:14.851638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:14.856337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:14.895480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f302-469d-4258-a1df-139d2a4d27aa service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:14.904705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:14.905636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:14.906132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:14.906654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] PATH_INFO: `/v3/users/29056329551c42bd8a42d752f40de3f3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:14.908189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29056329551c42bd8a42d752f40de3f3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:14.908665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 845/1687] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:14.909840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:14.911346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:14.911625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.020676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29056329551c42bd8a42d752f40de3f3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:15.073779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29056329551c42bd8a42d752f40de3f3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:15.076985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cd99e-a986-40bc-938d-5088443b1a5d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:15.093166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29056329551c42bd8a42d752f40de3f3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:15.109269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:15.119088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.119865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.120793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.122382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29056329551c42bd8a42d752f40de3f3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:15.124460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.124790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29056329551c42bd8a42d752f40de3f3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:15.144049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 843/1688] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:14 2022] DELETE /identity/v3/users/29056329551c42bd8a42d752f40de3f3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:15.145332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:15.149871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.153304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:15.155787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e3036-5270-47ac-88f0-52973c0ea529 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:15.177679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 846/1689] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:15.181091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.181509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.181812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.223465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:15.235726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] MFA Rules not processed for user `23126c9640d6433e96d393b885f3f829`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:15.240401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.318975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3837b9d-0698-47b5-828d-7c469684fea5 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:15.338994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:15.339896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.340201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.341674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.342403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:15.344795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:15.351356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801693ec-0e58-41bc-aa43-cc198d868fb0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:15.437968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 844/1690] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:15.446289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.446289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.451110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.456831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 847/1691] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:15.458678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:15.498995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf45b-b81a-44b5-8a4a-e0acbdc17438 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:15.517301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] MFA Rules not processed for user `6e3c1f7f6f8649a991d5d8a15eb8826b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:15.519420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:15.520184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.520571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.520951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.521514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.521835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:15.523907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:15.558662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:15.571811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 848/1692] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:15 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:15.574570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:15.580679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.691431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 845/1693] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:15.692358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:15.730589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e223-98a9-432e-8f97-0c047aaa3dc4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:15.781722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29063265-10eb-46fe-9543-b6fb50988154 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:15.799286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:15.800996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.801291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.801568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.807869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:15.892017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:15.892784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.893131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.893467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.894253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:15.911336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204d095-3a1a-46ec-b5bb-ebb9d29d4a25 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:15.979348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:15.981343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:15.983922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 846/1694] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:15 2022] POST /identity/v3/users => generated 291 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:15.985192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787413-2224-4d17-a95a-f59502053265 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:15.985481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787413-2224-4d17-a95a-f59502053265 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:15.985757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88787413-2224-4d17-a95a-f59502053265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:15.997683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 849/1695] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:16.009669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.037089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332b6c8-b931-47cb-bdb5-4b279ee8eba1 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.047169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.052256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.053098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.053386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.056524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.060288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:16.089713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 850/1696] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:16.091535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.098505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.106476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-88787413-2224-4d17-a95a-f59502053265 None None] MFA Rules not processed for user `f58a59c389cf4b3198a6c452eb2772d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:16.111799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-88787413-2224-4d17-a95a-f59502053265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.132109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57cdb5-202d-4805-b6e2-8d5541e5677c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.153945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.154794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.155442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.155442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] PATH_INFO: `/v3/projects/b8f53720176046919919835156882fff` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.159930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f53720176046919919835156882fff)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.163131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:16.192661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:16.271549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f53720176046919919835156882fff', 'request_id': 'req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:16.281359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f53720176046919919835156882fff', 'request_id': 'req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:16.303275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f53720176046919919835156882fff', 'request_id': 'req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:16.330105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] The token cache is being invalidate because project b8f53720176046919919835156882fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:16.330599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:16.331355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:16.337728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 851/1697] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:16 2022] DELETE /identity/v3/projects/b8f53720176046919919835156882fff => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:16.337728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.339408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.343739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 847/1698] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:16.345410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045248a0-f615-4eed-903c-c59ff6d58707 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.345786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045248a0-f615-4eed-903c-c59ff6d58707 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.346126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045248a0-f615-4eed-903c-c59ff6d58707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.381111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-045248a0-f615-4eed-903c-c59ff6d58707 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:16.382764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-045248a0-f615-4eed-903c-c59ff6d58707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.482310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:16.527543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf4e88-0758-4b11-bfb4-3d2fe3f2c402 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.530178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 848/1699] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:16.537094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.543127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.546668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.555385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.561630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.562934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.563203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.563992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.566649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:16.603564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 852/1700] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:16.604767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:16.607102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.608409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.629788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f781e-bd66-4f47-ab9b-c5047ae70a9f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.631233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 849/1701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:16.636908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.650390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03627f94-8d8b-4c53-9094-e2a4380e30c4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.650896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.651870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.652403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.652855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] PATH_INFO: `/v3/projects/15914d8976214f44836a8bfe4c9827d2/users/932bf4882fc344c49855cdb3605308ec/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.656178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.656803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.657315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.657589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.658911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:16.660502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.662805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:16.690147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15914d8976214f44836a8bfe4c9827d2, user_id=932bf4882fc344c49855cdb3605308ec, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.699196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47a043-b7c5-4004-9617-07937d7bcf5f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:16.702217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 850/1702] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:16.704500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:16.755426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43955579-44f1-41ab-8487-1a97f4ca3531 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:16.758015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 853/1703] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:16 2022] PUT /identity/v3/projects/15914d8976214f44836a8bfe4c9827d2/users/932bf4882fc344c49855cdb3605308ec/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:16.769597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2e29-7af1-4481-80a9-15b08d4f60b6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.770038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2e29-7af1-4481-80a9-15b08d4f60b6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.770421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2e29-7af1-4481-80a9-15b08d4f60b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.835352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e07b2e29-7af1-4481-80a9-15b08d4f60b6 None None] MFA Rules not processed for user `932bf4882fc344c49855cdb3605308ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:16.837134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e07b2e29-7af1-4481-80a9-15b08d4f60b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:16.939082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:16.939764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:16.939983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:16.940257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:16.940904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:16.942905 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcce48a-38a2-42b9-a7a3-abfbbebf33e5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.019937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 851/1704] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:16 2022] POST /identity/v3/projects => generated 365 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:17.019937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e425-b6b7-4b24-8912-1932499aae5b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.019937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e425-b6b7-4b24-8912-1932499aae5b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.020281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49e425-b6b7-4b24-8912-1932499aae5b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.029685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 852/1705] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 12 11:53:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:17.032944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:17.036963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e425-b6b7-4b24-8912-1932499aae5b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.047721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 854/1706] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:17.050963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.051255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.054452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.165301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:17.167289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.178417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49e425-b6b7-4b24-8912-1932499aae5b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.185627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.186274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.186585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.186864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.187510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.190488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.228553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 853/1707] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:17.230267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.232769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.252658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce57d7-3c8e-4402-9a08-aa70a68a8227 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.265883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.266826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.267206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.267578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] PATH_INFO: `/v3/projects/a06cc2bf74564d789428c1a2c96d789b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.268954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06cc2bf74564d789428c1a2c96d789b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.272903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.304390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 855/1708] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:17.327926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.343947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:17.346591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93dc41-d47c-4a52-bf0d-d03c4901de1c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.370913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06cc2bf74564d789428c1a2c96d789b', 'request_id': 'req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:17.396223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06cc2bf74564d789428c1a2c96d789b', 'request_id': 'req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:17.410650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06cc2bf74564d789428c1a2c96d789b', 'request_id': 'req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:17.438656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] The token cache is being invalidate because project a06cc2bf74564d789428c1a2c96d789b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:17.439118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:17.439900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4ef20f84-60ef-4d7f-bf25-75fbcf3f49a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:17.441641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 854/1709] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:17 2022] DELETE /identity/v3/projects/a06cc2bf74564d789428c1a2c96d789b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:17.444989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.444989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.444989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.446605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 855/1710] 10.222.0.15 () {54 vars in 912 bytes} [Mon Sep 12 11:53:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:17.448630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.451147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.481178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.482169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.482457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.482775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.483760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.486995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.540844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 856/1711] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:17.540844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.543230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.656592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157e04e-57de-4a34-a3cb-be617b7344c6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.700848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e68f60-7006-4189-b9b7-ba58f8b41814 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.710479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.712117 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.712616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.714109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.725003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.762446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.764148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2ae27-ee6b-4c6c-a40e-14e8774f88e0 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.789744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 857/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:17.791592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.796662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.797823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.798298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.798839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.799999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.801859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea252db-af00-4f77-8697-f4c5d5bb496e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.815009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.815297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.826582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 856/1713] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:17.830122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.830921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.831398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.875552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:17.878033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.881860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:17.901695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.902165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 857/1714] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:17.903358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.904884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.922121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947339f-280b-456d-baf0-7a051dfd7998 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.936375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 858/1715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:17.936604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.936718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.936805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.936805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.936805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.936805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.936805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.949678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d59140-6dcf-403a-a7c4-80a07df81809 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:17.961162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:17.962052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.965587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.965870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:17.968746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:17.972424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc06d9-f781-4577-8c35-dd9caf13274e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:17.975044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 858/1716] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/users => generated 296 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:17.975710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:17.985399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:17.991559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 859/1717] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:17.991559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:17.991559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:17.991559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.010017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:18.018441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:18.019093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd384f5-9dc4-4430-a20f-d7a82b0e62af admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.021277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:18.029290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.030406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.035850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 860/1718] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:18.035850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.036370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.036823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.046318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cb854-18d2-4c37-abc6-0e750dc8b3bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.052203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.052958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.053331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.053711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.054176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.054622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.056385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.058583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.069183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 861/1719] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:18.070152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.086368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7e02-ee4d-4767-b49b-fcf964b49807 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.087801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 859/1720] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:18.089005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.097129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.097821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.098100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.098373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] PATH_INFO: `/v3/projects/4a45a640be604b1cb58b564e4aee5957/users/c132b2b5deea4f8db19282864d13600b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.103408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd62bf1-8433-4c8d-8a3e-e4e737275c4d service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.118345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.119553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.120047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.120558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.121441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:18.121952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.123884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.140568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a45a640be604b1cb58b564e4aee5957, user_id=c132b2b5deea4f8db19282864d13600b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.145555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.146231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 860/1721] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:18.147904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.162384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954576-a0c2-4630-8901-a53b81d67b6f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.170645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.171727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.172214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.172720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] PATH_INFO: `/v3/projects/8ca6ddb7678348b0922300b51d134254/users/f863c76c468041d4b0f90b5455ecca85/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.176106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 862/1722] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:18 2022] PUT /identity/v3/projects/4a45a640be604b1cb58b564e4aee5957/users/c132b2b5deea4f8db19282864d13600b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:18.177077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.192148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:18.200541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca6ddb7678348b0922300b51d134254, user_id=f863c76c468041d4b0f90b5455ecca85, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.207223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbe333-5506-4697-94ff-cea3c4bf07f8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.240236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 861/1723] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:18 2022] PUT /identity/v3/projects/8ca6ddb7678348b0922300b51d134254/users/f863c76c468041d4b0f90b5455ecca85/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:18.245836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.246814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.248397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.313713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74a34f-33da-4bca-af4c-53f3e82e2510 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.355973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] MFA Rules not processed for user `c132b2b5deea4f8db19282864d13600b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:18.358110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:18.383790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.384496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.384732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.385005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.385646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.387888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.433725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 863/1724] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:18 2022] POST /identity/v3/projects => generated 389 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:18.437488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.506222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 862/1725] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:18.507110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.539200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127aa7a-6185-408d-aa0f-282e74e15be1 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.576711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:18.583206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc96a-9f5c-4627-8aac-3a62b2c74869 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.590146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.591406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.591406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.591406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.592783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.594895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.616718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 864/1726] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:18.619399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.645000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903594dc-ab38-4cda-8be7-41784fa6c08c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.662250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.663866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.664356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.664930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.665782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.683155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.755614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 865/1727] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:18 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:18.757637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.763295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.766542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.766861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.767150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.767858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.770472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.834622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 863/1728] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:18 2022] POST /identity/v3/users => generated 295 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:18.839763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:18.867391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1eb68-4ca3-4acb-a132-6ff245df6e29 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.937940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6acf-b30f-4101-92b3-4a0b3d9e411d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:18.943210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:18.944227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:18.944510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:18.944792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] PATH_INFO: `/v3/projects/8ca6ddb7678348b0922300b51d134254/users/f863c76c468041d4b0f90b5455ecca85/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:18.948214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca6ddb7678348b0922300b51d134254, user_id=f863c76c468041d4b0f90b5455ecca85, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:18.951789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:18.980294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 866/1729] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:18 2022] PUT /identity/v3/projects/8ca6ddb7678348b0922300b51d134254/users/f863c76c468041d4b0f90b5455ecca85/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:18.990196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.003069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.095508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.097790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.098077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.098485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.099322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.101358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.138183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:19.147351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 864/1730] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:18 2022] POST /identity/v3/users => generated 289 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:19.158876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.172437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f325db-219b-410a-ac91-bde06c614e3b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.182255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c6dc-8581-4a67-8664-5dc10fd586d3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.197694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.198417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.198753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.199078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.204454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.233271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.234945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f4418-7ad8-4a4a-a5bf-43186d72be7a service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.278969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 867/1731] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:19.280489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.280777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.281051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.286059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 868/1732] 10.222.0.15 () {54 vars in 961 bytes} [Mon Sep 12 11:53:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:19.288736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.295304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.325116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.325709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.326000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.326279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.326994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.328627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb64c0-c361-4529-a637-917ba67f444e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.358248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 865/1733] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:19.360361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.360727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.360936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.427278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] MFA Rules not processed for user `f863c76c468041d4b0f90b5455ecca85`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:19.428663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.479811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:19.573440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b603e0-19d4-48b1-b92b-1142df267aac None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.601029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.601812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.603820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.605412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.613174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.667124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 866/1734] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:19.668005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.683641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d972-fab6-4e92-9c10-8692336d4981 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.691685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.692862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.694883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.695521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.696844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.703212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65869afe-73bf-41dd-8b8a-87057c72257f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.737963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 867/1735] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:19.741028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.742798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.744139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.768948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.770974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.793808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 869/1736] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:19.795556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.799589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:19.801188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.815100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 868/1737] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Sep 12 11:53:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:19.816096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.818779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:19.846205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4feae-6ff7-414a-9bd8-a3f2d6f96c1c service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.865716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.866361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.866670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.866949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.867786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:19.876427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:19.903936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 870/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:19.930236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:19.964115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9f23d-d775-4f66-a84f-f5bec6ff4b0a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.972698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea34b8-8582-473c-905a-9f1371f68649 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:19.974130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.974880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.975389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.975389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] PATH_INFO: `/v3/projects/77df5378718d4b80af2eac0d557ab9ca/users/038a6dd1173b4aab86cc9c82f91a9aee/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:19.981940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:19.982591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:19.982874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:19.983146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.004028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:20.005492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:20.010808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77df5378718d4b80af2eac0d557ab9ca, user_id=038a6dd1173b4aab86cc9c82f91a9aee, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:20.013578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:20.040040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 869/1739] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:20.040957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:20.045977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 871/1740] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:19 2022] PUT /identity/v3/projects/77df5378718d4b80af2eac0d557ab9ca/users/038a6dd1173b4aab86cc9c82f91a9aee/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:20.046828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:20.099557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96db816-e5de-4fb9-81f4-476897f1b709 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:20.115280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:20.344295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911496-8769-4faa-b539-68876c494801 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:20.349876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:20.350544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:20.350961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:20.351126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.362162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:20.364614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:20.365440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:20.368452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] PATH_INFO: `/v3/projects/1a49b9af0d884636a2cd32eb43edf2e7/users/6a43973c73164da3b3afec89aad49466/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.390564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:20.396720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:20.483654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a49b9af0d884636a2cd32eb43edf2e7, user_id=6a43973c73164da3b3afec89aad49466, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:20.486988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 870/1741] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:20.489333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:20.499620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef0234-3ea7-45c6-92a2-d65c358eead9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:20.533618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca153d-f3f6-4577-8ba1-8747ca926f2b service placement] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:20.541984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 872/1742] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:20 2022] PUT /identity/v3/projects/1a49b9af0d884636a2cd32eb43edf2e7/users/6a43973c73164da3b3afec89aad49466/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:20.552759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363fc6a-fe4a-4487-ac6f-b12e6e805a95 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:20.552983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363fc6a-fe4a-4487-ac6f-b12e6e805a95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:20.553268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363fc6a-fe4a-4487-ac6f-b12e6e805a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.636226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c363fc6a-fe4a-4487-ac6f-b12e6e805a95 None None] MFA Rules not processed for user `66682d5656d3414eaebba666d4e0771f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:20.637426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c363fc6a-fe4a-4487-ac6f-b12e6e805a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:20.853210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:20.896939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:20.900090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:20.901328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:20.903196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.915728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:20.921289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:20.927964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 873/1743] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:20.927964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab209de-d5b3-48e8-b6dd-32cfb5e26bc7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:20.930933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab209de-d5b3-48e8-b6dd-32cfb5e26bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:20.932418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab209de-d5b3-48e8-b6dd-32cfb5e26bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:20.987377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 871/1744] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:20.998261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:21.043841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bab209de-d5b3-48e8-b6dd-32cfb5e26bc7 None None] MFA Rules not processed for user `6a43973c73164da3b3afec89aad49466`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:21.047384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bab209de-d5b3-48e8-b6dd-32cfb5e26bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:21.067631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f0ea8-2f7f-44bb-9f81-3b48f21e9cd3 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:21.162473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:21.169476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:21.172576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:21.176818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] PATH_INFO: `/v3/projects/77df5378718d4b80af2eac0d557ab9ca/users/038a6dd1173b4aab86cc9c82f91a9aee/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:21.191811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77df5378718d4b80af2eac0d557ab9ca, user_id=038a6dd1173b4aab86cc9c82f91a9aee, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:21.209958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:21.255342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 872/1745] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:20 2022] PUT /identity/v3/projects/77df5378718d4b80af2eac0d557ab9ca/users/038a6dd1173b4aab86cc9c82f91a9aee/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:21.258891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:21.337900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 874/1746] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:21.343088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:21.343194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:21.344202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:21.432847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] MFA Rules not processed for user `038a6dd1173b4aab86cc9c82f91a9aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:21.436520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:21.618896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a721c35-0d91-42d2-97f7-69d3d8d23222 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:21.641824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 875/1747] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:21.642803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:21.643292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f863c76c468041d4b0f90b5455ecca85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ca6ddb7678348b0922300b51d134254', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:21.644096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:21.644545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:21.644960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:21.646121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:21.647649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:21.662866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:21.704991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 873/1748] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:21.706309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:21.756068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:21.888210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023412a4-6dfc-4a6f-9735-d6e0a6608c5c tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:21.892987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:21.894351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:21.894989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:21.895428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:21.907060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b010ae-4b71-4587-8ab8-7d8140d6adad None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:21.922750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:21.926886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:21.926886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:21.927174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:21.927459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:21.927724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:21.928851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:21.941751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:21.952675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 874/1749] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:21.954770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:21.982556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:21.988547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46e668-f5c7-4d50-8a04-3ec880a3778f service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.000983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f863c76c468041d4b0f90b5455ecca85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ca6ddb7678348b0922300b51d134254', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.001875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.002500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.003115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.003902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.006113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.063085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 876/1750] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:21 2022] POST /identity/v3/projects => generated 387 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:22.064018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:22.069273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 875/1751] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:21 2022] POST /identity/v3/projects => generated 354 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:22.074781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:22.358086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bddb-4cf3-49e5-9be2-502980aca9d3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.391816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14835c3-ed77-423c-bfd3-a4f1144668d8 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.411720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f863c76c468041d4b0f90b5455ecca85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ca6ddb7678348b0922300b51d134254', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.412465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.415451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.415875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.419438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:22.419896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.421647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.462629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 876/1752] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:22.647395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.648048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.648351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.648615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.649512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.656014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.709837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:22.721100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 877/1753] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:22 2022] POST /identity/v3/users => generated 300 bytes in 657 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:22.723050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9cdcf-f0c4-40f2-8134-f70f3fc4e8f0 tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.724424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:22.743914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6465e5b-094a-4f5c-8da1-5c9f0655be0d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.750990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f863c76c468041d4b0f90b5455ecca85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ca6ddb7678348b0922300b51d134254', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.751811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.752248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.752641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] PATH_INFO: `/v3/projects/f1d49f1f2c1d400fbae91343d23d17cf` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.758904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f1d49f1f2c1d400fbae91343d23d17cf)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.766373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.785698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.786684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.787489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.787992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.789181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.793399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.821890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 878/1754] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:22.841839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:22.877622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eb1fc-a7a4-4f38-b856-5878309a3d45 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:22.894942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:22.895913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:22.896318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:22.896718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] PATH_INFO: `/v3/projects/3e692bb5eccc426b94d3a091400e24cd/users/cea78af5bbf34646ad3bc803f4a31bbf/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:22.946452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d49f1f2c1d400fbae91343d23d17cf', 'request_id': 'req-6930088d-b4cc-43ce-9d68-5834cca9b32a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:22.948138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e692bb5eccc426b94d3a091400e24cd, user_id=cea78af5bbf34646ad3bc803f4a31bbf, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:22.953958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:22.960976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d49f1f2c1d400fbae91343d23d17cf', 'request_id': 'req-6930088d-b4cc-43ce-9d68-5834cca9b32a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:22.971931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d49f1f2c1d400fbae91343d23d17cf', 'request_id': 'req-6930088d-b4cc-43ce-9d68-5834cca9b32a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:22.996717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] The token cache is being invalidate because project f1d49f1f2c1d400fbae91343d23d17cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:22.997130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:22.998090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.001241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 877/1755] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:22 2022] DELETE /identity/v3/projects/f1d49f1f2c1d400fbae91343d23d17cf => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:23.010031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 879/1756] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:22 2022] PUT /identity/v3/projects/3e692bb5eccc426b94d3a091400e24cd/users/cea78af5bbf34646ad3bc803f4a31bbf/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:23.014196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:23.015912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:23.043640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:23.322620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930088d-b4cc-43ce-9d68-5834cca9b32a tempest-VolumeQuotaClassesTest-480059120 tempest-VolumeQuotaClassesTest-480059120-project-admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:23.541409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:23.547656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:23.562054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:23.565979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:23.567491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:23.568838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:23.569672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:23.578800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:23.590426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd5a1-c385-492b-b525-121f6d9b6a95 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:23.607531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:23.608205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:23.608665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:23.608944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] PATH_INFO: `/v3/users/c132b2b5deea4f8db19282864d13600b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:23.610359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c132b2b5deea4f8db19282864d13600b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:23.612316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:23.624800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 878/1757] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 608 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:23.625999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:23.650467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f429852-1828-426d-a640-63eaaf4f8cde admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:23.662261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:23.667278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:23.667793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:23.668307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] PATH_INFO: `/v3/projects/3e692bb5eccc426b94d3a091400e24cd/users/cea78af5bbf34646ad3bc803f4a31bbf/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:23.671922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e692bb5eccc426b94d3a091400e24cd, user_id=cea78af5bbf34646ad3bc803f4a31bbf, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:23.692250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:23.729880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:23.730984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 879/1758] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:23 2022] PUT /identity/v3/projects/3e692bb5eccc426b94d3a091400e24cd/users/cea78af5bbf34646ad3bc803f4a31bbf/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:23.733122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:23.747622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:23.749945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c132b2b5deea4f8db19282864d13600b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.790819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c132b2b5deea4f8db19282864d13600b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.833152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c132b2b5deea4f8db19282864d13600b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.833921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c132b2b5deea4f8db19282864d13600b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.857231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0ee7e4c9-ed9d-4937-9f28-1dc6083e814e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c132b2b5deea4f8db19282864d13600b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:23.872063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 880/1759] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:23 2022] DELETE /identity/v3/users/c132b2b5deea4f8db19282864d13600b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:23.873629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9b41-da72-462c-9252-6d0d40f83cff None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:23.874098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9b41-da72-462c-9252-6d0d40f83cff None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:23.874373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9b41-da72-462c-9252-6d0d40f83cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:23.906402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70fef5-4ed0-4ff4-aec0-d9b8b281321a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:23.913057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-12ad9b41-da72-462c-9252-6d0d40f83cff None None] MFA Rules not processed for user `cea78af5bbf34646ad3bc803f4a31bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:23.914641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-12ad9b41-da72-462c-9252-6d0d40f83cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.037355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:24.111193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:24.114036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.114325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.116472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] PATH_INFO: `/v3/users/932bf4882fc344c49855cdb3605308ec` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.122145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=932bf4882fc344c49855cdb3605308ec)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:24.128363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:24.255683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 881/1760] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:24.264260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.264582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.264990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.282878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932bf4882fc344c49855cdb3605308ec'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:24.312500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932bf4882fc344c49855cdb3605308ec'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:24.317193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:24.323540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.337698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932bf4882fc344c49855cdb3605308ec'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:24.350162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932bf4882fc344c49855cdb3605308ec'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:24.353451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b8ca473-ca6a-4269-bce0-f25e548cd442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932bf4882fc344c49855cdb3605308ec'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:24.370811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 880/1761] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:23 2022] DELETE /identity/v3/users/932bf4882fc344c49855cdb3605308ec => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:24.372438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b457-f803-4732-8840-8dc71d878c1d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.373916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b457-f803-4732-8840-8dc71d878c1d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.373916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b457-f803-4732-8840-8dc71d878c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.375797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 882/1762] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:24.376578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:24.386740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.468328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ea79b457-f803-4732-8840-8dc71d878c1d None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:24.470569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ea79b457-f803-4732-8840-8dc71d878c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.543010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b392c-1b0e-4e17-ba9a-0d5e0a2237bc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:24.636734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 881/1763] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:53:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:24.643613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.644286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.645670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.668556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:24.668797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.669086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.669503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.671802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.683665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:24.683994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:24.685085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.727544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:24.764454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 882/1764] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:24.765476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:24.767371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.822876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:24.824947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:24.845829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 883/1765] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:24.860323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:24.864301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.913910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7ff16-3b07-4028-9bc2-889d31d6e74d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:24.935836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:24.936447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:24.937715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:24.938006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:24.947564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:24.996339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:24.997864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:25.036300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f3049-d9ab-4bcc-958a-23e95ac70e4e service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:25.050036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 883/1766] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:25.056180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:25.059315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:25.059958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.060242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.062769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.068363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.103861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:25.105965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:25.119851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 884/1767] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:25.122355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:25.124464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.141912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc232c-af08-4d6e-92a7-4640c762f0d7 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:25.162414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:25.163272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.164595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.164992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] PATH_INFO: `/v3/users/7c3dbb2ca04745b7bdb8baf40c45a649` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.168173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3dbb2ca04745b7bdb8baf40c45a649)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:25.174888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:25.249238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3dbb2ca04745b7bdb8baf40c45a649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.261432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3dbb2ca04745b7bdb8baf40c45a649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.284964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788584cc-e188-4532-8126-446bbaadf15e service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:25.298692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:25.299091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3dbb2ca04745b7bdb8baf40c45a649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.301323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.301587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3dbb2ca04745b7bdb8baf40c45a649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.301778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.302062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.305472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.317104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4f89fba7-9ba3-4402-9c58-c208246773a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3dbb2ca04745b7bdb8baf40c45a649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.318631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:25.320489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:25.332250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 884/1768] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:53:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:53:25.333472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:25.335389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.352834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9064-ec9e-42b0-b5fe-7bde7ca661f2 service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:25.358978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 885/1769] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:25 2022] DELETE /identity/v3/users/7c3dbb2ca04745b7bdb8baf40c45a649 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:25.364607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6657e1-5122-4129-8789-97c5254424c2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.365009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6657e1-5122-4129-8789-97c5254424c2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.365663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6657e1-5122-4129-8789-97c5254424c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.407391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9f6657e1-5122-4129-8789-97c5254424c2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:25.409227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9f6657e1-5122-4129-8789-97c5254424c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.426135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:25.498047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:25.520205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:25.520751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 886/1770] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:25.523397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.524735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.525318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] PATH_INFO: `/v3/users/6a43973c73164da3b3afec89aad49466` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.526966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a43973c73164da3b3afec89aad49466)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:25.530062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:25.612945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.613778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.614329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.635174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a43973c73164da3b3afec89aad49466'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.652784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] MFA Rules not processed for user `f58a59c389cf4b3198a6c452eb2772d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:25.665034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.687354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a43973c73164da3b3afec89aad49466'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.713192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a43973c73164da3b3afec89aad49466'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.733258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a43973c73164da3b3afec89aad49466'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.735228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18301a71-88db-43ed-b00a-88b715a6840f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a43973c73164da3b3afec89aad49466'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:25.752247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 885/1771] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:25 2022] DELETE /identity/v3/users/6a43973c73164da3b3afec89aad49466 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:25.759557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:25.760340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:25.762117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:25.807870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 887/1772] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:25.835919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:25.839068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:25.940923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:25.943209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:26.066243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 886/1773] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:26.075076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d2f1-8a09-422d-932b-4868d07f9f42 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:26.198384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:26.199075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:26.199347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:26.199624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:26.201539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:26.228288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:26.229772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31fa0-f1df-4c01-b221-6b122aff50e1 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:26.256056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 888/1774] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:26.274879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:26.277975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:26.323273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fee90-8938-4b0a-97dd-a9d96e7e34e2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:26.337417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:26.338471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:26.338962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:26.340397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:26.375352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:26.383393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:26.409473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 887/1775] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:27.473408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:27.474666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:27.475282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:27.514481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:27.519499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:27.568801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 889/1776] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 12 11:53:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:27.597692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:27.603758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:27.639778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388c274-3ab2-467c-974d-a66605e9fb0d service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:27.651285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:27.652001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:27.652286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:27.652568 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] PATH_INFO: `/v3/projects/15914d8976214f44836a8bfe4c9827d2` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:27.663303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15914d8976214f44836a8bfe4c9827d2)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:27.672468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:27.806991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:27.817159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a827cd-1d35-41fe-8750-e203a9dc8eb9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:27.838880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15914d8976214f44836a8bfe4c9827d2', 'request_id': 'req-03fa2224-767e-4cad-8a7b-5756457ec564'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:27.850981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:27.857572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:27.860243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15914d8976214f44836a8bfe4c9827d2', 'request_id': 'req-03fa2224-767e-4cad-8a7b-5756457ec564'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:27.882423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15914d8976214f44836a8bfe4c9827d2', 'request_id': 'req-03fa2224-767e-4cad-8a7b-5756457ec564'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:27.909394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] The token cache is being invalidate because project 15914d8976214f44836a8bfe4c9827d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:27.909741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:27.912686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:27.918103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 888/1777] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:27 2022] DELETE /identity/v3/projects/15914d8976214f44836a8bfe4c9827d2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:27.931445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:27.933748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:27.972426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:27.975346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:27.976322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:27.977419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:27.981381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:28.011570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:28.015311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:28.043988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 890/1778] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:28.122245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2224-767e-4cad-8a7b-5756457ec564 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:28.142053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:28.142808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:28.143089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:28.143359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] PATH_INFO: `/v3/users/f863c76c468041d4b0f90b5455ecca85` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:28.146464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f863c76c468041d4b0f90b5455ecca85)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:28.148732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:28.200026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:28.204305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:28.234251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbb5b-a00d-471d-9043-89fd54e79b3f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:28.248400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:28.249176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:28.249554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:28.249969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] PATH_INFO: `/v3/projects/4a45a640be604b1cb58b564e4aee5957` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:28.252391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a45a640be604b1cb58b564e4aee5957)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:28.257125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:28.292001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f863c76c468041d4b0f90b5455ecca85'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.327170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f863c76c468041d4b0f90b5455ecca85'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.365176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f863c76c468041d4b0f90b5455ecca85'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.394093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f863c76c468041d4b0f90b5455ecca85'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.396895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f863c76c468041d4b0f90b5455ecca85'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.441072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 889/1779] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:27 2022] DELETE /identity/v3/users/f863c76c468041d4b0f90b5455ecca85 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:28.462141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a45a640be604b1cb58b564e4aee5957', 'request_id': 'req-556dd012-8c53-4a17-98a6-159c908c7712'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.472668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a45a640be604b1cb58b564e4aee5957', 'request_id': 'req-556dd012-8c53-4a17-98a6-159c908c7712'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.510648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a45a640be604b1cb58b564e4aee5957', 'request_id': 'req-556dd012-8c53-4a17-98a6-159c908c7712'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.547241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] The token cache is being invalidate because project 4a45a640be604b1cb58b564e4aee5957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:28.547673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.549143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:28.557142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 891/1780] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:28 2022] DELETE /identity/v3/projects/4a45a640be604b1cb58b564e4aee5957 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:28.563368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:28.567156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:28.772512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc1853-c805-4f3d-ac7c-281504671db4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:28.801084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:28.801734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:28.802103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:28.802430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] PATH_INFO: `/v3/users/038a6dd1173b4aab86cc9c82f91a9aee` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:28.804584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038a6dd1173b4aab86cc9c82f91a9aee)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:28.809560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:28.819384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:28.982726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038a6dd1173b4aab86cc9c82f91a9aee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:29.048297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038a6dd1173b4aab86cc9c82f91a9aee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:29.082558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038a6dd1173b4aab86cc9c82f91a9aee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:29.118546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038a6dd1173b4aab86cc9c82f91a9aee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:29.121287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038a6dd1173b4aab86cc9c82f91a9aee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:29.158565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 890/1781] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:28 2022] DELETE /identity/v3/users/038a6dd1173b4aab86cc9c82f91a9aee => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:30.076499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:30.080232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:30.229863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dd012-8c53-4a17-98a6-159c908c7712 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:30.419344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:30.457288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:30.458210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:30.458666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:30.459227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] PATH_INFO: `/v3/projects/1a49b9af0d884636a2cd32eb43edf2e7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:30.463320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a49b9af0d884636a2cd32eb43edf2e7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:30.465465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:30.572833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a49b9af0d884636a2cd32eb43edf2e7', 'request_id': 'req-37bfad41-5149-466e-890a-80142f267ae4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:30.599275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a49b9af0d884636a2cd32eb43edf2e7', 'request_id': 'req-37bfad41-5149-466e-890a-80142f267ae4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:30.617318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a49b9af0d884636a2cd32eb43edf2e7', 'request_id': 'req-37bfad41-5149-466e-890a-80142f267ae4'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:30.662020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] The token cache is being invalidate because project 1a49b9af0d884636a2cd32eb43edf2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:30.662300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:30.663890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:30.669246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 892/1782] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:30 2022] DELETE /identity/v3/projects/1a49b9af0d884636a2cd32eb43edf2e7 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:30.671087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:30.672841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:30.698251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f0e48-1f53-432d-bc91-d7b6df3339c0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:30.712768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:30.714707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:30.885006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:30.905668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:30.906312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:30.906758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] PATH_INFO: `/v3/users/cea78af5bbf34646ad3bc803f4a31bbf` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:30.915055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea78af5bbf34646ad3bc803f4a31bbf)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:30.920283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:30.950747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfad41-5149-466e-890a-80142f267ae4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:30.963180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:30.964045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:30.964525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:30.964910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] PATH_INFO: `/v3/projects/aeedeefa0bd94ad180b19f0d559c9563` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:30.967246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeedeefa0bd94ad180b19f0d559c9563)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:30.969627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:31.057533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea78af5bbf34646ad3bc803f4a31bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.112378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeedeefa0bd94ad180b19f0d559c9563', 'request_id': 'req-fe0a91a2-5a8b-47ab-b5cd-418a83024980'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.122177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeedeefa0bd94ad180b19f0d559c9563', 'request_id': 'req-fe0a91a2-5a8b-47ab-b5cd-418a83024980'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.126056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea78af5bbf34646ad3bc803f4a31bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.155466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeedeefa0bd94ad180b19f0d559c9563', 'request_id': 'req-fe0a91a2-5a8b-47ab-b5cd-418a83024980'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.172228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea78af5bbf34646ad3bc803f4a31bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.194301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea78af5bbf34646ad3bc803f4a31bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.195244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea78af5bbf34646ad3bc803f4a31bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.196349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] The token cache is being invalidate because project aeedeefa0bd94ad180b19f0d559c9563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:31.197072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.198420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:31.199044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fe0a91a2-5a8b-47ab-b5cd-418a83024980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.202837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 893/1783] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:30 2022] DELETE /identity/v3/projects/aeedeefa0bd94ad180b19f0d559c9563 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:31.207619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c7fd0-7951-4203-b112-d257257008e5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.208078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c7fd0-7951-4203-b112-d257257008e5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.208561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c7fd0-7951-4203-b112-d257257008e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.227497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 891/1784] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:30 2022] DELETE /identity/v3/users/cea78af5bbf34646ad3bc803f4a31bbf => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:31.265689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cc7c7fd0-7951-4203-b112-d257257008e5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:31.267101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cc7c7fd0-7951-4203-b112-d257257008e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:31.276400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:31.422363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:31.423957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:31.459539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 894/1785] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:31.468013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.473047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.473595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.516065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:31.517879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:31.555504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 895/1786] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:31.557259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:31.581417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764ec3-dc98-4a8c-ae39-ac1dc7a60d34 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:31.596838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:31.597491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.597784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.598368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.600497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:31.601678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:31.603991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:31.613749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e4c95-2134-4fe2-9a1d-14ebb6a81794 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:31.633344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 896/1787] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:31.633786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:31.634454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:31.634960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.635400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.635816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] PATH_INFO: `/v3/projects/77df5378718d4b80af2eac0d557ab9ca` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.637919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77df5378718d4b80af2eac0d557ab9ca)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:31.640206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:31.647931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f468a-3735-49b1-b29b-d570ee54d271 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:31.655652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:31.656322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.656582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.656993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.657643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:31.659703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:31.704013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 897/1788] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:31 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:31.705660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:31.812610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77df5378718d4b80af2eac0d557ab9ca', 'request_id': 'req-a1e5e35c-ca41-4620-b506-e42640fdf82e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.829060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77df5378718d4b80af2eac0d557ab9ca', 'request_id': 'req-a1e5e35c-ca41-4620-b506-e42640fdf82e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.838062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77df5378718d4b80af2eac0d557ab9ca', 'request_id': 'req-a1e5e35c-ca41-4620-b506-e42640fdf82e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.844561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688f25-2c11-45d5-b027-31964f51fccd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:31.884157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] The token cache is being invalidate because project 77df5378718d4b80af2eac0d557ab9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:31.885117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.886085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1e5e35c-ca41-4620-b506-e42640fdf82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:31.888193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 892/1789] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:31 2022] DELETE /identity/v3/projects/77df5378718d4b80af2eac0d557ab9ca => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:31.892875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaeab0-850f-4dd2-8112-498fd2ea0cdd None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:31.893466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaeab0-850f-4dd2-8112-498fd2ea0cdd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:31.893956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaeab0-850f-4dd2-8112-498fd2ea0cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:31.948195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0deaeab0-850f-4dd2-8112-498fd2ea0cdd None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:31.951783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0deaeab0-850f-4dd2-8112-498fd2ea0cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:32.128002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.128794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.129181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.129561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.130315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.132389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.202235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 893/1790] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:32.205102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 898/1791] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:31 2022] POST /identity/v3/users => generated 285 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:32.209555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.210175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.210843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.215203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.219424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:32.248478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:32.267290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73890a-dff6-4e18-957f-0b64867dccb4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.281690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:32.293272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.294015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.294329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.294640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.295430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.297170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.306456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:32.334708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 894/1792] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:32.340630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.364813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 899/1793] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:32.365020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ee098-dfcf-4acd-b1a3-5ea0c9ce6c8f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.371095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.393444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.400091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.400390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.402815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.405794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:32.406235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.411696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.415002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c3610-0293-4dde-b0bf-b6d159df926d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.428368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.433322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.433686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.434016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.434901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.437903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.449247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 895/1794] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:32.450210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.484676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 900/1795] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:32.487477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.490793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aafeba-18af-4227-90f7-3db9f5567577 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.509001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.510693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.513264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.513855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.515461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.544690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0105-71b0-46cf-b313-d4e5b6ea3e43 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.546139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.566721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.568860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.569141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.569405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] PATH_INFO: `/v3/projects/397c81e0065b4d2289720855ad8da5f7/users/167cc6d2e68c49ada52ecc6764c832eb/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.599927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:32.658496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397c81e0065b4d2289720855ad8da5f7, user_id=167cc6d2e68c49ada52ecc6764c832eb, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.669726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.669940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 896/1796] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:32 2022] POST /identity/v3/projects => generated 365 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:32.669940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.676864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:32.714566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 901/1797] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:32 2022] PUT /identity/v3/projects/397c81e0065b4d2289720855ad8da5f7/users/167cc6d2e68c49ada52ecc6764c832eb/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:32.729849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:32.918727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee528-25dd-4096-be94-3efaab51db71 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.931769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.932427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.932709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.932997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.933716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.947227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe55c29-ebc6-444f-9e4b-1432303ae727 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:32.962331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb42bf-90c8-4b73-8992-865548fa8964 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:32.980514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:32.982332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:32.982643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:32.982922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] PATH_INFO: `/v3/projects/8ca6ddb7678348b0922300b51d134254` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:32.985730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca6ddb7678348b0922300b51d134254)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:32.993072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.019826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 902/1798] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:32 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:33.021277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ff32-a6f0-4979-a6ce-814c948b8273 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.030354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ff32-a6f0-4979-a6ce-814c948b8273 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.030679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ff32-a6f0-4979-a6ce-814c948b8273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.159284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6fe8ff32-a6f0-4979-a6ce-814c948b8273 None None] MFA Rules not processed for user `167cc6d2e68c49ada52ecc6764c832eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:33.160604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6fe8ff32-a6f0-4979-a6ce-814c948b8273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:33.184901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca6ddb7678348b0922300b51d134254', 'request_id': 'req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:33.200644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca6ddb7678348b0922300b51d134254', 'request_id': 'req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:33.210070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca6ddb7678348b0922300b51d134254', 'request_id': 'req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:33.236153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] The token cache is being invalidate because project 8ca6ddb7678348b0922300b51d134254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:33.237030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:33.237356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:33.239548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 897/1799] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:32 2022] DELETE /identity/v3/projects/8ca6ddb7678348b0922300b51d134254 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:33.244798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.252993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:33.365405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 903/1800] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:33.378322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.380082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.380651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.464711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:33.471090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:33.499325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783b229-c5a6-4a89-8bc4-4260fc0bf34e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:33.502857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 904/1801] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:33.503766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.509848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:33.515248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:33.516961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.517094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.517303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.520146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:33.521943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e55fc-7409-47ec-ab08-2b513500d97c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.552988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20133c4-882f-4171-b533-a876fb7a9628 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:33.561800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 898/1802] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:33.565003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.567728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.568057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.582270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:33.584874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.585221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.585677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.586379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:33.614095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.635473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:33.637810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:33.671420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 899/1803] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:33.672662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.695873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 905/1804] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:33.695873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.699112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ccfe6-4af7-42b9-8aca-7b5a8f766fc7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:33.719308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:33.727662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:33.860695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51ceb2-78d6-4e48-b5d2-9efa3da1ad4c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:33.875237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:33.877487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.878325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.879395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.881764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:33.882081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:33.888855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.924076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 906/1805] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:33.925075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.939185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0999-6364-41ee-9b85-011a67d7175a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:33.948696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:33.949297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.953182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.953460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.954137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:33.954277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:33.955404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:33.955979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:33.956662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:33.958684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:33.959138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.960756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:33.977596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 900/1806] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:33.979646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:33.998410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3006-97e7-448d-bd24-32b4c8d6f8e9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.006068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.007276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 907/1807] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/users => generated 293 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:34.008995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.009483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.010020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.010487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.011468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.015839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6019-c946-45f5-b8cc-2c5f2db861c9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.039784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c93a2-f924-4d16-a3eb-560d4a173889 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.046310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.047084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.047395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.047672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] PATH_INFO: `/v3/projects/8ff58c676c4e4026954679daa616a04d/users/b4c915511d4d44a180cd3e340dee84cb/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.063065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 901/1808] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:33 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:34.066431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afec9-f94b-40b8-ab32-df354b85f38a None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.066953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afec9-f94b-40b8-ab32-df354b85f38a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.067389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afec9-f94b-40b8-ab32-df354b85f38a None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.069505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 902/1809] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:34.070472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9afec9-f94b-40b8-ab32-df354b85f38a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.082295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff58c676c4e4026954679daa616a04d, user_id=b4c915511d4d44a180cd3e340dee84cb, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.087577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.114009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 908/1810] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:34 2022] PUT /identity/v3/projects/8ff58c676c4e4026954679daa616a04d/users/b4c915511d4d44a180cd3e340dee84cb/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:34.115292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.220180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9afec9-f94b-40b8-ab32-df354b85f38a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.314911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a137-55ef-4bb4-b464-ea2247b2a3e2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.335564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.336533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.339097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.339570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.340483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.352199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a675de-f33d-4041-8a47-283c764f2662 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.405531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.406672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.407170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.410108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.410991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.417596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.422238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 909/1811] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:34 2022] POST /identity/v3/users => generated 294 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:34.426232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88950df0-8836-4d9f-91ce-bcd80a00a073 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.426557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88950df0-8836-4d9f-91ce-bcd80a00a073 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.428191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88950df0-8836-4d9f-91ce-bcd80a00a073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.453016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 903/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:34.453016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.473668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31183d-f347-41ad-b3dd-d5e651b4a076 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.484119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.485019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.485411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.485806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.486626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.490071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.496796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-88950df0-8836-4d9f-91ce-bcd80a00a073 None None] MFA Rules not processed for user `b4c915511d4d44a180cd3e340dee84cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:34.498355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-88950df0-8836-4d9f-91ce-bcd80a00a073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:34.521810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 904/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:34.521810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.547020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f02a68-d116-4704-82b4-cd19139e3b16 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.562077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:34.562288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.563212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.564190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.564190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] PATH_INFO: `/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8/users/ec497855204e4e3cb82af745a8f52e41/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.595371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:34.599276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dd27a2e9b84e51926f7f2eb394c8e8, user_id=ec497855204e4e3cb82af745a8f52e41, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.603875 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.621595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 910/1814] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:34.623725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.624182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.624692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.634559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 905/1815] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:34 2022] PUT /identity/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8/users/ec497855204e4e3cb82af745a8f52e41/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:34.635665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.658274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:34.660868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:34.771227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e435fb-8334-469e-9e3b-82fc5b753e40 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.783348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.784121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.784467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.785179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.786186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.788067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.802340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 906/1816] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:34.803848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.826987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cfcbf-fcde-4c69-9ee5-a18fb0bbdb1b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.830023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 911/1817] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:53:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:34.837566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.851802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.852727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.855001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.855001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.855001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.857558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.863891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c25b4-9f99-492b-ab08-e16cea0906a8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.872060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.873614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.874156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.874642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.876226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.878127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:34.896071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 912/1818] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:34.897392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.923610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068d44-6c20-4432-8fc9-b06ecc309574 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:34.935271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:34.936252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:34.936574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:34.936915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] PATH_INFO: `/v3/projects/cf608944fad34ec08f005e50c4850f9f/users/5558973a74ee47f68132671191e74952/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:34.959929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 907/1819] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:34 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:34.962307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:34.966619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf608944fad34ec08f005e50c4850f9f, user_id=5558973a74ee47f68132671191e74952, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:34.969838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:35.001425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 913/1820] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:34 2022] PUT /identity/v3/projects/cf608944fad34ec08f005e50c4850f9f/users/5558973a74ee47f68132671191e74952/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:35.003331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:35.139811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ae23-a643-4e38-ac20-bd212ed613e9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:35.155407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001548db-2f9b-425f-ae1f-d049d88a34fb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:35.169463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:35.170530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.170530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.171258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.172229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:35.176255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7d421-b042-4e51-a775-984f0849b084 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:35.177756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:35.179096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.179919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.180360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] PATH_INFO: `/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8/users/ec497855204e4e3cb82af745a8f52e41/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.184832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dd27a2e9b84e51926f7f2eb394c8e8, user_id=ec497855204e4e3cb82af745a8f52e41, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:35.188869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:35.219925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 908/1821] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:34 2022] PUT /identity/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8/users/ec497855204e4e3cb82af745a8f52e41/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:35.221282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:35.224893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:35.231033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 914/1822] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:35 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:35.240099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ec100-359e-4ac5-a9fd-ea25d0fbae3f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.240099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ec100-359e-4ac5-a9fd-ea25d0fbae3f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.240099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ec100-359e-4ac5-a9fd-ea25d0fbae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.336867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f14ec100-359e-4ac5-a9fd-ea25d0fbae3f None None] MFA Rules not processed for user `5558973a74ee47f68132671191e74952`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:35.339312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f14ec100-359e-4ac5-a9fd-ea25d0fbae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:35.340983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4061-5013-4644-b561-323a37e6d872 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:35.347679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:35.348621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.350223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.350572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] PATH_INFO: `/v3/projects/3e692bb5eccc426b94d3a091400e24cd` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.354553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e692bb5eccc426b94d3a091400e24cd)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:35.357048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:35.358306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:35.372715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:35.489463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e692bb5eccc426b94d3a091400e24cd', 'request_id': 'req-36e17586-63a7-428e-90ae-994210dea736'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:35.525869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 915/1823] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:35.527352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abea83-3f63-4291-8052-7514a705fed3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.528435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abea83-3f63-4291-8052-7514a705fed3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.530158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abea83-3f63-4291-8052-7514a705fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.547990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e692bb5eccc426b94d3a091400e24cd', 'request_id': 'req-36e17586-63a7-428e-90ae-994210dea736'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:35.566737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e692bb5eccc426b94d3a091400e24cd', 'request_id': 'req-36e17586-63a7-428e-90ae-994210dea736'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:35.587290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] The token cache is being invalidate because project 3e692bb5eccc426b94d3a091400e24cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:35.587778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:35.588582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:35.592265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 909/1824] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:35 2022] DELETE /identity/v3/projects/3e692bb5eccc426b94d3a091400e24cd => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:35.593549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:35.598287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:35.643210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-69abea83-3f63-4291-8052-7514a705fed3 None None] MFA Rules not processed for user `ec497855204e4e3cb82af745a8f52e41`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:35.644453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-69abea83-3f63-4291-8052-7514a705fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:35.833510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e17586-63a7-428e-90ae-994210dea736 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:35.849269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:35.850624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.850945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.852951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.853636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:35.860130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:35.901239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 910/1825] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:35.903821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:35.909370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:35.941329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 916/1826] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:35.944413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26e2d0-f665-456a-81a8-82089bcccec1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.944752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26e2d0-f665-456a-81a8-82089bcccec1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.946486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26e2d0-f665-456a-81a8-82089bcccec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.956794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421e5a4-d694-4db3-b30a-c62b9432bba2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:35.972444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:35.974841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:35.975301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:35.976472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:35.978675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:35.986156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:36.012665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ec26e2d0-f665-456a-81a8-82089bcccec1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:36.015351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ec26e2d0-f665-456a-81a8-82089bcccec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.046764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 917/1827] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:36.048690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840678-c214-43c0-87a6-e654dea75686 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.049090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840678-c214-43c0-87a6-e654dea75686 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.049537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840678-c214-43c0-87a6-e654dea75686 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.075936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 918/1828] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:53:36 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:36.077687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840678-c214-43c0-87a6-e654dea75686 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:36.088221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 911/1829] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:35 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:36.090656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:36.096691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.264171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840678-c214-43c0-87a6-e654dea75686 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:36.269248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4a72-4ba7-4547-a05b-ac2b8d7b2350 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:36.276048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:36.277634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.277982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.278329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] PATH_INFO: `/v3/projects/cb3b795f4ff84529bec718f90e009a9b/users/7281679da82a45b3b432ad6ee6930c0d/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.281378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:36.282350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.282868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.283412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.325513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:36.365094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3b795f4ff84529bec718f90e009a9b, user_id=7281679da82a45b3b432ad6ee6930c0d, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:36.367876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaa09c-045d-4406-9b7b-ea9c93f0266e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:36.410952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 919/1830] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:36 2022] PUT /identity/v3/projects/cb3b795f4ff84529bec718f90e009a9b/users/7281679da82a45b3b432ad6ee6930c0d/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:36.415085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.415372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.415645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.489711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:36.491015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.506300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:36.509187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:36.543843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:36.546861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 912/1831] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:36.548818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:36.580392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb6841-248b-4402-951d-7a5cb4e47738 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:36.709682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 920/1832] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:36.710926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:36.712685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.768440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:36.769076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.769354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.769626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.773162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:36.777498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33c256-0ad6-4b2b-9538-004c9d89a0ad admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:36.832158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 913/1833] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:36 2022] POST /identity/v3/users => generated 299 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:36.834761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.835566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.835966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.877105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:36.878475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.922718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ead6b-1ad7-4d84-98ad-f24aa3feca83 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:36.938761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:36.941131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:36.942851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:36.944872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:36.949080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:36.982621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:36.989332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.018276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 914/1834] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:53:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:37.019412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.024618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 921/1835] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:37.029344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.051069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15780a6c-dc73-421f-b8e6-99c14247a8d8 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.060679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493dc5-2d20-45e6-a4d4-0d36d98ae46b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.065592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.066216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.067007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.067007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.067563 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:37.067887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.070984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.093332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.094109 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.094396 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.094730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.095380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.097021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.097712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 922/1836] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:37.108121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.147002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 915/1837] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:37.150552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.178720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8673c-bdfb-42ac-8cbd-8f9d7838f464 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.197000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.198154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.198784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.199358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.200314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.204442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.241538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 916/1838] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:37.241888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8989e-c68f-4bfd-8bec-330092ee5b1d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.244952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.251384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.252311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.252441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.252671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.259418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:37.281835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b7cd-efb7-4a74-ba07-c73fd3880a5f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.297924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.299064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.299363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.299643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.300296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.303016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.319370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:37.355201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 917/1839] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:37 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:37.356130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.370136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:37.395422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.397142 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.417010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 923/1840] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:37.418229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.433325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3cfe-6df0-4433-aaac-6d4a0ce60713 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.491481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86104-c6c0-4a63-bdbb-99d90298f363 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.504481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.506357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.509367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.509661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] PATH_INFO: `/v3/projects/cb3b795f4ff84529bec718f90e009a9b/users/7281679da82a45b3b432ad6ee6930c0d/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.516222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3b795f4ff84529bec718f90e009a9b, user_id=7281679da82a45b3b432ad6ee6930c0d, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.519185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.520181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.520580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.521225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] PATH_INFO: `/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc/users/55c4a431a17242ed811ee2c4ce2c4088/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.522766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.545327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f316a2039b584ef7b076a9cd0ae60dbc, user_id=55c4a431a17242ed811ee2c4ce2c4088, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.547831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 918/1841] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:37 2022] PUT /identity/v3/projects/cb3b795f4ff84529bec718f90e009a9b/users/7281679da82a45b3b432ad6ee6930c0d/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:37.555958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.556082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466bc86-cb9a-4487-9bcd-7938a6a1426f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.579748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 924/1842] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:37 2022] PUT /identity/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc/users/55c4a431a17242ed811ee2c4ce2c4088/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:37.584067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.584429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.585236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.635029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] MFA Rules not processed for user `7281679da82a45b3b432ad6ee6930c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:37.636721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:37.770119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ebb88-ea81-4ce6-821a-8ad4911d9bc6 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.792565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 925/1843] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:37.793804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.887323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.887854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.888134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.888404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.889035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.891190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.892476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:37.908596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c43030-a969-4723-a3f9-aa618b93462e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:37.915196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:37.915808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:37.916085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:37.916359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:37.916723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 919/1844] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:37 2022] POST /identity/v3/users => generated 305 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:37.917635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.919985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:37.921585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:37.935268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 926/1845] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:37.935268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:37.945182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:37.998732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9a478-f8c1-4b9f-824f-3919b0de29db admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.002693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.003449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.003568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.004223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.009014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b69ce-d555-448f-8b8a-986f06d7ff24 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.015217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.015728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.016347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.016625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.016725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.016911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.028010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.029809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.030690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 927/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:38.032051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.045873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 920/1847] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:38.046754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.052516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd32824-f4d5-48c9-8bac-7987028ae74b service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.060524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7e30f-8b4c-40f1-9ff9-9514ee3b7916 service nova] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.065996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.066713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.066994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.067269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] PATH_INFO: `/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc/users/55c4a431a17242ed811ee2c4ce2c4088/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.069951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f316a2039b584ef7b076a9cd0ae60dbc, user_id=55c4a431a17242ed811ee2c4ce2c4088, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.074494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7450-4dbd-4ad0-9732-feae93c4f91e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.096917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 921/1848] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:38 2022] PUT /identity/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc/users/55c4a431a17242ed811ee2c4ce2c4088/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:38.107965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.108615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.108886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.110777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.110998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.111114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.111325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.111995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.113736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.129439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 928/1849] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:38.134037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.153031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b746917-2232-42c5-a495-5fc04a053ccb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.153226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] MFA Rules not processed for user `55c4a431a17242ed811ee2c4ce2c4088`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:38.155273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:38.226780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.227709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.228115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.228483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.229228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.231078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.237826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 922/1850] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:38.238876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.245918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 929/1851] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:38.247141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.258708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769f9a6-1151-428c-bb33-71d80b4251a9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.264350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.265281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.265640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.266010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] PATH_INFO: `/v3/projects/a0567c935a4f4e7daf4a327cd2b6ed84/users/b942d90f319f4ead8c138177ffbe731e/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.284014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0567c935a4f4e7daf4a327cd2b6ed84, user_id=b942d90f319f4ead8c138177ffbe731e, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.286754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.303305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 930/1852] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:38 2022] PUT /identity/v3/projects/a0567c935a4f4e7daf4a327cd2b6ed84/users/b942d90f319f4ead8c138177ffbe731e/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:38.304797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.306708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:38.319057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d9e58-c00e-44de-b182-38ef451f838f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.372985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:38.390911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404452a7-91fb-4d10-99ad-0cb54d6e991f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.396505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.397564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.398143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.398896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.399502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.400201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.400603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.400983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.402949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:38.405381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:38.470437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.472010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.472500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.473956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d978f-b65e-45d9-950b-a6805892cf99 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.484679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 923/1853] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:38.485474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 931/1854] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:38.485968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.486256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.486578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.519247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] MFA Rules not processed for user `b942d90f319f4ead8c138177ffbe731e`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:38.521202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:38.621332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 924/1855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:38.623119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.762360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7ddd5-7aa5-49b9-aec8-eec459c81714 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.770538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.771199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.774424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.774933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.775619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.778132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.840175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 932/1856] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:38 2022] POST /identity/v3/projects => generated 409 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:38.840276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:38.849854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27af54d-cc0f-43ba-a9db-be07d35a04dc None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:38.938137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:38.938137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:38.938137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:38.938137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:38.938554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:38.938554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:38.986455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 925/1857] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:38 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:38.987694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.005865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245391c-b94f-4488-90fc-0c0797051f0c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.017176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.017933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.018220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.018500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.019186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:39.023555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:39.059810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 933/1858] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:39.064758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.103202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ff21b-47ba-454c-b28b-49ad5e6d21a1 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.138547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.141501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.141790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.142067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] PATH_INFO: `/v3/projects/70965608305e4564a90b63bb341bffa1/users/ebb2aab34cbe4e5eb92c70fab49cbb9e/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.167617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:39.213486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70965608305e4564a90b63bb341bffa1, user_id=ebb2aab34cbe4e5eb92c70fab49cbb9e, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:39.222927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:39.269585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 926/1859] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:39 2022] PUT /identity/v3/projects/70965608305e4564a90b63bb341bffa1/users/ebb2aab34cbe4e5eb92c70fab49cbb9e/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:39.271032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.287015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b848b-d565-4b7f-ae25-9dc8be188869 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.483776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.484469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.484680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.484983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.485623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:39.487258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0192c0-643e-4df9-8ed2-fddbe4ec1eef admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:39.517801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 934/1860] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:39.520607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.541061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791b1-15cd-4845-9b8d-ec87a031bb38 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.555040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.555735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.556019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.556297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] PATH_INFO: `/v3/projects/70965608305e4564a90b63bb341bffa1/users/ebb2aab34cbe4e5eb92c70fab49cbb9e/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.560650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70965608305e4564a90b63bb341bffa1, user_id=ebb2aab34cbe4e5eb92c70fab49cbb9e, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:39.563366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:39.600910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 927/1861] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:39 2022] PUT /identity/v3/projects/70965608305e4564a90b63bb341bffa1/users/ebb2aab34cbe4e5eb92c70fab49cbb9e/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:39.603235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.603632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.603909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.633456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.678862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:39.689416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] MFA Rules not processed for user `ebb2aab34cbe4e5eb92c70fab49cbb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:39.694420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:39.754829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146715-021c-480f-8453-f1bc39f4f182 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.764421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.765093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.766332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.766786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] PATH_INFO: `/v3/users/5558973a74ee47f68132671191e74952` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.768337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5558973a74ee47f68132671191e74952)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:39.770263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:39.809425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 935/1862] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:39 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:39.825370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:39.840514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5558973a74ee47f68132671191e74952'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:39.863062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5558973a74ee47f68132671191e74952'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:39.874226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5558973a74ee47f68132671191e74952'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:39.890647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5558973a74ee47f68132671191e74952'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:39.891828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9b319fce-d8af-4d6a-9e1e-3d11dd41974f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5558973a74ee47f68132671191e74952'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:39.898617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 928/1863] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:39 2022] DELETE /identity/v3/users/5558973a74ee47f68132671191e74952 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:39.903121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d482f99-027c-464d-b335-119b61bf123a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.903419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d482f99-027c-464d-b335-119b61bf123a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.903866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d482f99-027c-464d-b335-119b61bf123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.915094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bad6b-af61-407a-a3cd-e5110ee052aa None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:39.930957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4d482f99-027c-464d-b335-119b61bf123a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:39.932718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4d482f99-027c-464d-b335-119b61bf123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:39.982740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:39.983191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:39.983484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:39.983764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:39.986118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:39.990491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:40.014664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 929/1864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:40.027340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d482f99-027c-464d-b335-119b61bf123a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:40.028981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4d482f99-027c-464d-b335-119b61bf123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:40.073540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:40.077419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:40.096691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 936/1865] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:40.104094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d482f99-027c-464d-b335-119b61bf123a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:40.108878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:40.109546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:40.109785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:40.110896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:40.126915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:40.128433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:40.139770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 930/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:40.310330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:40.318701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:40.333916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:40.341386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413d50-65b7-49f0-9964-e2bb78c3ed39 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:40.347367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:40.348422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:40.348801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:40.349514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:40.351825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:40.448448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:40.449811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:40.460230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 937/1867] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:40.506458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:40.508222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:40.522303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec68cf-64b3-400a-a327-05109a1bab6c service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:40.531725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:40.532457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:40.532825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:40.533209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] PATH_INFO: `/v3/projects/cf608944fad34ec08f005e50c4850f9f` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:40.534437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf608944fad34ec08f005e50c4850f9f)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:40.536585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:40.589860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf608944fad34ec08f005e50c4850f9f', 'request_id': 'req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.597011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf608944fad34ec08f005e50c4850f9f', 'request_id': 'req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.605135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf608944fad34ec08f005e50c4850f9f', 'request_id': 'req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.617829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] The token cache is being invalidate because project cf608944fad34ec08f005e50c4850f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:40.618122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.618796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.620297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 931/1868] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:40 2022] DELETE /identity/v3/projects/cf608944fad34ec08f005e50c4850f9f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:40.624510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:40.636153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566bb6a-2512-4c66-aa0b-23bc07f28398 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:40.698277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:40.698792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:40.699091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:40.699407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] PATH_INFO: `/v3/users/55c4a431a17242ed811ee2c4ce2c4088` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:40.700881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c4a431a17242ed811ee2c4ce2c4088)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:40.703887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:40.781332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c4a431a17242ed811ee2c4ce2c4088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.795020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c4a431a17242ed811ee2c4ce2c4088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.826554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c4a431a17242ed811ee2c4ce2c4088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.827582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c4a431a17242ed811ee2c4ce2c4088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.836598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-06791af8-0447-4f20-8220-af36ecb8798a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c4a431a17242ed811ee2c4ce2c4088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:40.846253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 938/1869] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:40 2022] DELETE /identity/v3/users/55c4a431a17242ed811ee2c4ce2c4088 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:41.300225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.302231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.321286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bc4fe-1d7d-4b78-8415-26835a8b3c1b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.379005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.379634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.379911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.380190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] PATH_INFO: `/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.381291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f316a2039b584ef7b076a9cd0ae60dbc)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.383385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.404169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:41.431130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f316a2039b584ef7b076a9cd0ae60dbc', 'request_id': 'req-8203bb6b-de67-43e9-a48e-2c99c713370b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.438953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f316a2039b584ef7b076a9cd0ae60dbc', 'request_id': 'req-8203bb6b-de67-43e9-a48e-2c99c713370b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.444777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f316a2039b584ef7b076a9cd0ae60dbc', 'request_id': 'req-8203bb6b-de67-43e9-a48e-2c99c713370b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.456514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] The token cache is being invalidate because project f316a2039b584ef7b076a9cd0ae60dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:41.456799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.457481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.459235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 932/1870] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:41 2022] DELETE /identity/v3/projects/f316a2039b584ef7b076a9cd0ae60dbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:41.465016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972817-1d1d-4509-8946-19c9e9571b0c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.465385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972817-1d1d-4509-8946-19c9e9571b0c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.465662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972817-1d1d-4509-8946-19c9e9571b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.487840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-65972817-1d1d-4509-8946-19c9e9571b0c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:41.488960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-65972817-1d1d-4509-8946-19c9e9571b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.506680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:41.552265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 939/1871] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:41.553766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.555919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.566261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.566584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.566868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.569556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203bb6b-de67-43e9-a48e-2c99c713370b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.575602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.576385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.576777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.577147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] PATH_INFO: `/v3/users/b4c915511d4d44a180cd3e340dee84cb` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.578573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4c915511d4d44a180cd3e340dee84cb)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.580692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.593708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:41.594624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.604272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 940/1872] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:41.608524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.619739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c14c-7079-411f-aa0a-e455fae4e057 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.624688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.625401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.625750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.626093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.626863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:41.627237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.628927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.635737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4c915511d4d44a180cd3e340dee84cb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.649533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 941/1873] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:53:41.653626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.667324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e899-f1ff-4278-be23-c8e77ce7465d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.669891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4c915511d4d44a180cd3e340dee84cb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.679745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4c915511d4d44a180cd3e340dee84cb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.680280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.681568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.681859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.682140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.682835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.684771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.689423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4c915511d4d44a180cd3e340dee84cb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.691132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-d0854db8-858f-4a22-92fe-b283fadb7f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4c915511d4d44a180cd3e340dee84cb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:41.699146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 933/1874] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:41 2022] DELETE /identity/v3/users/b4c915511d4d44a180cd3e340dee84cb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:41.703242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.703527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.703804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.721338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 942/1875] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:41 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:41.725426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.727101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:41.727755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.728288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.799186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797ffb-ce33-4418-b209-605f26e0b932 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.803513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 934/1876] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:41.805516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.806145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.806483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.806827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.807434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.809375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.833488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 943/1877] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:41 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:53:41.834063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.837926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:41.839070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.854866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c9e2a-ad46-47f8-8fbb-b384cf2191ed admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.860064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.860918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.861376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.861834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.862706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.864553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.873192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:41.882556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 944/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:41.887605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.902576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec67d-3eff-4da1-9934-8e76a9781dbd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.908438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.909065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.909492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.909720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.910371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.912453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.922237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1ea3c-12ac-4412-a9e8-da3e4e0d585e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.926526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.927422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.927583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 945/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:41.927829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.928110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.932510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:41.950177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:41.955187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937eb09-d69a-4ebe-ab04-df8c48e4c1fb service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:41.970563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 935/1880] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:41.972733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13215-75b0-41a3-b982-e9d8c8d38a99 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:41.978748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:41.979428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:41.979705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:41.979976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] PATH_INFO: `/v3/projects/af3d6f40eb2948dab8a824d2859e884a/users/324046b7a16a42fcb27d72622f6affd6/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:41.997389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3d6f40eb2948dab8a824d2859e884a, user_id=324046b7a16a42fcb27d72622f6affd6, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.000062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:42.018601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 946/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:41 2022] PUT /identity/v3/projects/af3d6f40eb2948dab8a824d2859e884a/users/324046b7a16a42fcb27d72622f6affd6/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:42.028969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.029310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.029587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.079061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] MFA Rules not processed for user `324046b7a16a42fcb27d72622f6affd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:42.082341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:42.203708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 936/1882] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:42.220419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:42.223217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:42.270111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:42.311885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94cd46-d4ef-4949-af99-b1d5f571dced admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:42.316871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:42.317576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.317937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.318286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.332692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.334718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22c66e-6315-4448-943c-e1dcc733e942 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:42.347654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 947/1883] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:42.377373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:42.484363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c3ff2-911a-4295-acec-50bc8cb2ab53 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:42.489536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:42.490453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.490570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.490834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] PATH_INFO: `/v3/users/324046b7a16a42fcb27d72622f6affd6` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.492238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324046b7a16a42fcb27d72622f6affd6)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.494347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:42.544369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324046b7a16a42fcb27d72622f6affd6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.549615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:42.557648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324046b7a16a42fcb27d72622f6affd6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.571002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324046b7a16a42fcb27d72622f6affd6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.580200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324046b7a16a42fcb27d72622f6affd6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.580897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324046b7a16a42fcb27d72622f6affd6'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.590472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 937/1884] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:42 2022] DELETE /identity/v3/users/324046b7a16a42fcb27d72622f6affd6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:42.594885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.595404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.595756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.618664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:42.620036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:42.683602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 948/1885] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:42.693345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:42.695825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:42.729970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:42.731630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:42.745579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93628f4b-9897-4e2a-bd06-21849561163d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:42.751063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:42.752264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.752550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.752826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] PATH_INFO: `/v3/projects/8ff58c676c4e4026954679daa616a04d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.754028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff58c676c4e4026954679daa616a04d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.756037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:42.782211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3af6-9846-4e2c-9604-ef94cefe5ef9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:42.787710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:42.788605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.789132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.789642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.801032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.803088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:42.814435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 938/1886] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:42.831693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff58c676c4e4026954679daa616a04d', 'request_id': 'req-b78c193e-44b6-4441-9a31-9311d5eaab89'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.838823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff58c676c4e4026954679daa616a04d', 'request_id': 'req-b78c193e-44b6-4441-9a31-9311d5eaab89'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.845811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff58c676c4e4026954679daa616a04d', 'request_id': 'req-b78c193e-44b6-4441-9a31-9311d5eaab89'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.861593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] The token cache is being invalidate because project 8ff58c676c4e4026954679daa616a04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:42.862092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.862982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:42.864870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 949/1887] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:42 2022] DELETE /identity/v3/projects/8ff58c676c4e4026954679daa616a04d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:42.869122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:42.880074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea780c-d760-4b9b-95ad-d6f34adcae92 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:42.943798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:42.944467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:42.944761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:42.945038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] PATH_INFO: `/v3/users/7281679da82a45b3b432ad6ee6930c0d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:42.946821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7281679da82a45b3b432ad6ee6930c0d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:42.948909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.010384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7281679da82a45b3b432ad6ee6930c0d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.026942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7281679da82a45b3b432ad6ee6930c0d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.036901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7281679da82a45b3b432ad6ee6930c0d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.043182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:43.046558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7281679da82a45b3b432ad6ee6930c0d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.047857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-57db4cf6-d74c-412b-9b49-6d8a9ace9053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7281679da82a45b3b432ad6ee6930c0d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.056405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 939/1888] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:53:42 2022] DELETE /identity/v3/users/7281679da82a45b3b432ad6ee6930c0d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:43.162832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.165085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.178635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c193e-44b6-4441-9a31-9311d5eaab89 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.239195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:43.274493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.275292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.275617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.275891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] PATH_INFO: `/v3/projects/af3d6f40eb2948dab8a824d2859e884a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.277364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3d6f40eb2948dab8a824d2859e884a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.279521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.329285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3d6f40eb2948dab8a824d2859e884a', 'request_id': 'req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.334989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3d6f40eb2948dab8a824d2859e884a', 'request_id': 'req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.342096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3d6f40eb2948dab8a824d2859e884a', 'request_id': 'req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.361488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] The token cache is being invalidate because project af3d6f40eb2948dab8a824d2859e884a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:43.362165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.363198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.365118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 950/1889] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:43 2022] DELETE /identity/v3/projects/af3d6f40eb2948dab8a824d2859e884a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:43.370597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56cbc-5bb9-4ac8-a5ab-333780592c7c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.370841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56cbc-5bb9-4ac8-a5ab-333780592c7c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.371157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b56cbc-5bb9-4ac8-a5ab-333780592c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.393208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c6b56cbc-5bb9-4ac8-a5ab-333780592c7c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:43.395069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.395947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c6b56cbc-5bb9-4ac8-a5ab-333780592c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.398245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.467421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 940/1890] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:43.474728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b43df-0dee-49c9-8f1f-97ebf4cb909a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.474935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.475221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.475502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.481370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.482176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.486629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.487276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] PATH_INFO: `/v3/projects/cb3b795f4ff84529bec718f90e009a9b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.489303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3b795f4ff84529bec718f90e009a9b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.492106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.498383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:43.499716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.514107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 941/1891] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:43.515036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.525587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb92992-9ddf-4fe2-8e33-c3181c392277 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.532013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.532297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.532579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.532852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.533506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:43.533812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.535806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.550829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 942/1892] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:53:43.555695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.568327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a1661-3679-4ca3-b3bb-25b09b741b1b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.576506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3b795f4ff84529bec718f90e009a9b', 'request_id': 'req-800e212c-c052-48a7-9a4a-61e4569a2813'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.587109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3b795f4ff84529bec718f90e009a9b', 'request_id': 'req-800e212c-c052-48a7-9a4a-61e4569a2813'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.595127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3b795f4ff84529bec718f90e009a9b', 'request_id': 'req-800e212c-c052-48a7-9a4a-61e4569a2813'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.612092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] The token cache is being invalidate because project cb3b795f4ff84529bec718f90e009a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:53:43.612572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.613625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-800e212c-c052-48a7-9a4a-61e4569a2813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:53:43.615869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 951/1893] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:53:43 2022] DELETE /identity/v3/projects/cb3b795f4ff84529bec718f90e009a9b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:43.621074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7606-435e-41f2-9ad8-bf62bb675b4b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.621434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7606-435e-41f2-9ad8-bf62bb675b4b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.625281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7606-435e-41f2-9ad8-bf62bb675b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.647832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d0df7606-435e-41f2-9ad8-bf62bb675b4b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:43.649160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d0df7606-435e-41f2-9ad8-bf62bb675b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.655146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:43.660471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.662778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.663065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.663874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.664648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.666687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.701518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 943/1894] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:43.705387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.707129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.732917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 952/1895] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:43.742219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.743100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.743390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.767544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:43.768899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:43.778299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 953/1896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:43.782742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.784132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b287779-5881-46c0-b982-b65a7899137f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.790663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.791552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.791991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.792404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.793227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.793452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eade8-27d0-45a3-8fa4-58e74526ad43 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.795485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.798139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.798781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.799065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.799338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.799997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:53:43.800305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.801938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.805736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:43.821218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 954/1897] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:53:43.823709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.829825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 944/1898] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/users => generated 285 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:43.831089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.835931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2af41-8cac-48c3-b1a6-bb2a16015637 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.840774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.841724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.842097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ce81-ddbc-4096-b4c5-30534f6b645d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.842197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.842665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.843725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.845997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.854452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.855200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.855554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.855898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.856928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.858238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.870217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 945/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:53:43.874284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.878149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 955/1900] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:43.878934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.884726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7b9d8-f9f7-4fe5-a80d-b38b59bc7d39 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.942236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.943039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.943443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.943812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.945110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.945578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ffe75-c02c-44aa-b82a-18c3e0ead8b3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.947174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.950584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.951194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.951477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.951756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.952390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.954298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:43.957000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 946/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:43.961241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.971377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21f203-648a-4d1d-ae83-6e4d880f9a8e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.976733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.977393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:43.977673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:43.981259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] PATH_INFO: `/v3/projects/e270a066f14d43069b2d5867d01fb373/users/45aec9064c4e444a917f52d2f947461b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:43.981379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 956/1902] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:43 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:43.982018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:43.993996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63922f44-a131-4a30-b3de-4f149ac86928 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:43.997620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e270a066f14d43069b2d5867d01fb373, user_id=45aec9064c4e444a917f52d2f947461b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:43.999614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:43.999872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.000222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.000329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.000604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.001311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.003252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968102e-041c-4f22-b601-d23238c058e1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.014927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 947/1903] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:43 2022] PUT /identity/v3/projects/e270a066f14d43069b2d5867d01fb373/users/45aec9064c4e444a917f52d2f947461b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:44.018123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 957/1904] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:44.019228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.025742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.026134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.026311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.060682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] MFA Rules not processed for user `45aec9064c4e444a917f52d2f947461b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:44.062137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:44.089743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688062eb-594b-4dc6-a2e7-99dc364b626d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.095003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.095615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.096210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.096210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.096866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.098698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.114270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 948/1905] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:53:44.114489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:44.116214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.128735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f4798-31e4-4769-81af-ce65fbf19890 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.134046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.134733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.135011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.135401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] PATH_INFO: `/v3/projects/4187a5ed9e0a44aea442b8fb59baf2b5/users/fecdd60f06af4878a56b19e572ab9086/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.153003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 958/1906] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:44.156236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4187a5ed9e0a44aea442b8fb59baf2b5, user_id=fecdd60f06af4878a56b19e572ab9086, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.159577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fe7d2-2c25-405a-9428-ed7112628821 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.166443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.167986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:44.175800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 949/1907] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:44 2022] PUT /identity/v3/projects/4187a5ed9e0a44aea442b8fb59baf2b5/users/fecdd60f06af4878a56b19e572ab9086/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:44.186365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.186879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.187248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.215416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] MFA Rules not processed for user `fecdd60f06af4878a56b19e572ab9086`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:44.216633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:44.227395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:44.253260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba82def-8202-4176-b84b-436dd2855bb8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.286436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 950/1908] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:53:44.291579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.311787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.312406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.312692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.313093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.360426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0cd3-f48e-47f5-a5b3-db226972269e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.365186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.365793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.366078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.366928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.367779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.369679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.371585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.373021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.383940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 959/1909] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:53:44.399713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 951/1910] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:53:44 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:44.400814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.417437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34ec92-9752-4c10-b2f0-41cb6c0d6169 service glance] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.459354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:44.495116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.495727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.496003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.496272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.496905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.498857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.524030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 960/1911] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:53:44 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:53:44.525134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.536336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb527-1407-41d4-8893-440463c423f1 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.541238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.541865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.542124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.542395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.543087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.545318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.555297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 952/1912] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:53:44.559611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.569888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4be1b9-42c6-4a88-9fae-fa64ddfe148b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.575296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.576200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.576593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.576871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] PATH_INFO: `/v3/projects/35b3209cb5094dac94fb9b7d759d975c/users/95c33ab80ea14b25b874cb61420dbc40/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.592865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b3209cb5094dac94fb9b7d759d975c, user_id=95c33ab80ea14b25b874cb61420dbc40, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.595627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.609747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 961/1913] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:44 2022] PUT /identity/v3/projects/35b3209cb5094dac94fb9b7d759d975c/users/95c33ab80ea14b25b874cb61420dbc40/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:53:44.614023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.624503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244226-5afc-47c9-85ee-d9a936d33cf0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.686998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.687757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.688140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.688517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.689304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.691197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72275-b8ec-4384-9175-82be134c33f3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.702564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 953/1914] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:53:44.706973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.721883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13df5a8-8f55-4c83-b94b-5ed249ef4d9d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:44.726885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:44.727632 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.728642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.728642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] PATH_INFO: `/v3/projects/35b3209cb5094dac94fb9b7d759d975c/users/95c33ab80ea14b25b874cb61420dbc40/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.730493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b3209cb5094dac94fb9b7d759d975c, user_id=95c33ab80ea14b25b874cb61420dbc40, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:44.733524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:44.749565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 962/1915] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:53:44 2022] PUT /identity/v3/projects/35b3209cb5094dac94fb9b7d759d975c/users/95c33ab80ea14b25b874cb61420dbc40/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:53:44.761253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ef97-41fd-4883-83bf-fa44b71d4071 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:44.761536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ef97-41fd-4883-83bf-fa44b71d4071 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:44.761907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ef97-41fd-4883-83bf-fa44b71d4071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:44.804692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d424ef97-41fd-4883-83bf-fa44b71d4071 None None] MFA Rules not processed for user `95c33ab80ea14b25b874cb61420dbc40`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:44.805213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d424ef97-41fd-4883-83bf-fa44b71d4071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:44.815416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:44.913783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 954/1916] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:53:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:44.931539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:44.939969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:45.017476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7e260-8365-49d2-97e8-49534328a225 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:45.021128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:45.021738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:45.022080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:45.022351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:45.088059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:45.089575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccc63f-faa2-4198-a65f-6ff67b3ab53a service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:45.098538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 963/1917] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:53:53.230259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:53.231639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:53.232125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:53.250185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 955/1918] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:53:53 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:53:53.258754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:53.259300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:53.259686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:53.294442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:53:53.296721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:53.335358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:53:53.390021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 964/1919] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:53:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:53:53.402782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:53:53.404807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:53:53.425235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831db30-ae24-4b42-929c-953e6fb77e6d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:53:53.435022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:53:53.436044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:53:53.436337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:53:53.436613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:53:53.447410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:53:53.449522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:53:53.474044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 956/1920] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:53:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:00.280344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:00.290278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef94890-897f-406d-80aa-f48d03281793 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:00.294435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:00.295092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:00.295374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:00.295642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:00.305652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:00.307727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9d49d-f16f-485d-be02-0f119d2b33a8 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:00.318779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 965/1921] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:00.794311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:00.795962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:00.865002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36acfa-f68b-440e-b21e-16c7d5d321f5 service nova] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:00.870718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:00.871364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:00.871649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:00.871937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] PATH_INFO: `/v3/users/b942d90f319f4ead8c138177ffbe731e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:00.873081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b942d90f319f4ead8c138177ffbe731e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:00.875039 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:00.921481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b942d90f319f4ead8c138177ffbe731e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:00.934415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b942d90f319f4ead8c138177ffbe731e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:00.942294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b942d90f319f4ead8c138177ffbe731e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:00.949958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b942d90f319f4ead8c138177ffbe731e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:00.950620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b942d90f319f4ead8c138177ffbe731e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:00.956404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 957/1922] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:00 2022] DELETE /identity/v3/users/b942d90f319f4ead8c138177ffbe731e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:00.961375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:00.961664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:00.961971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:00.981536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:00.982900 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:01.042832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 966/1923] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:54:01.052489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:01.053925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:01.068454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:01.114171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aef60b-f5b5-45a3-8f22-e5b65a35815e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:01.117938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:01.119148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:01.119430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:01.119694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:01.128785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:01.130209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:01.139503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 958/1924] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:01.460009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:01.461673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:01.474469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc8339-fde8-46db-aedf-8ae948164911 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:01.481079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:01.481728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:01.482002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:01.482273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] PATH_INFO: `/v3/projects/a0567c935a4f4e7daf4a327cd2b6ed84` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:01.486092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0567c935a4f4e7daf4a327cd2b6ed84)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:01.488070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:01.548143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0567c935a4f4e7daf4a327cd2b6ed84', 'request_id': 'req-03909328-32ef-4233-9aa4-553ce9c44f14'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.555226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0567c935a4f4e7daf4a327cd2b6ed84', 'request_id': 'req-03909328-32ef-4233-9aa4-553ce9c44f14'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.562308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0567c935a4f4e7daf4a327cd2b6ed84', 'request_id': 'req-03909328-32ef-4233-9aa4-553ce9c44f14'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.579913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] The token cache is being invalidate because project a0567c935a4f4e7daf4a327cd2b6ed84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:01.580263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.586772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.592176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 967/1925] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:01 2022] DELETE /identity/v3/projects/a0567c935a4f4e7daf4a327cd2b6ed84 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:01.593658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:01.608469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950c0a-eea6-40b7-8f5e-0722ecbe4113 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:01.698893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:01.699529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:01.699808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:01.700073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] PATH_INFO: `/v3/users/ebb2aab34cbe4e5eb92c70fab49cbb9e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:01.701175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb2aab34cbe4e5eb92c70fab49cbb9e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:01.703237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:01.750805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb2aab34cbe4e5eb92c70fab49cbb9e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.765952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb2aab34cbe4e5eb92c70fab49cbb9e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.775830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb2aab34cbe4e5eb92c70fab49cbb9e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.795964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb2aab34cbe4e5eb92c70fab49cbb9e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.796975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-aaf50ce7-3ffc-4fcd-b592-76d1afc4b702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb2aab34cbe4e5eb92c70fab49cbb9e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:01.804471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 959/1926] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:01 2022] DELETE /identity/v3/users/ebb2aab34cbe4e5eb92c70fab49cbb9e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:54:02.167712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.169151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:02.184606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03909328-32ef-4233-9aa4-553ce9c44f14 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.219528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:02.246465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.247162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.247460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.247732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] PATH_INFO: `/v3/projects/70965608305e4564a90b63bb341bffa1` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.248846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70965608305e4564a90b63bb341bffa1)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.250801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.296784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70965608305e4564a90b63bb341bffa1', 'request_id': 'req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:02.302146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70965608305e4564a90b63bb341bffa1', 'request_id': 'req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:02.309424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70965608305e4564a90b63bb341bffa1', 'request_id': 'req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:02.321408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] The token cache is being invalidate because project 70965608305e4564a90b63bb341bffa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:02.321694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:02.322551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3f72bfae-265f-4b64-bcd2-8c81a5a6e059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:02.324032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 968/1927] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:02 2022] DELETE /identity/v3/projects/70965608305e4564a90b63bb341bffa1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:02.328861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.329172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.329451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.349210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:02.350410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:02.380916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:02.410827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 960/1928] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:02.419001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.419288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.419564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.439614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:02.440814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:02.448924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 969/1929] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:02.453097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.462818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c0706-5e5c-4ff8-a675-cebdf5b256e3 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.467335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.467928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.468209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.468479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.469116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:54:02.469417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.471038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.483105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 961/1930] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:54:02.487315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.497242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96084-812a-4f60-a28c-a99c7e3f8953 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.501918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.502569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.502879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.503150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.503790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.505703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.532988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 970/1931] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:02.537134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.547750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab404-a251-4e7a-9f49-5974592d3061 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.603145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.603711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.603919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.604219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.604847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.606790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.628930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 962/1932] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:02.633116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.643347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70c3bb-512d-45e0-a1a0-c1093517a624 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.647905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.648502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.648781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.649057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.650194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.652091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.661883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 971/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:02.666464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.676290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861d11-2e8e-409b-bf34-6bd1275fc10a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.681167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.681935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.682220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.682501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.683255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.684871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066c5b7-71eb-45b4-8973-fcfef76bce8f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.696438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 963/1934] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:02.700832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.717910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158b57-8fc5-4906-890f-1b0814962dbc admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.721544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.722207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.722569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.722853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] PATH_INFO: `/v3/projects/237073c227614875859f9b6eb4b6bce4/users/971c1d5226434eef816e2bd248fecd8b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.741423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=237073c227614875859f9b6eb4b6bce4, user_id=971c1d5226434eef816e2bd248fecd8b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.744241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.761103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 972/1935] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:02 2022] PUT /identity/v3/projects/237073c227614875859f9b6eb4b6bce4/users/971c1d5226434eef816e2bd248fecd8b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:02.773787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.774152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.778424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.815123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] MFA Rules not processed for user `971c1d5226434eef816e2bd248fecd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:02.817160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:02.904146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 964/1936] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:02.908611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:02.970859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332a76b-8dad-4b80-b77a-6a16406039ed admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:02.975680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:02.976283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:02.976561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:02.976969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:02.978148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:02.980074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:02.993087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:03.029518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 973/1937] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:02 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:03.032808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.044888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247368-a0a8-464a-a0a9-bfba8202987e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.081483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:03.124783 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.125407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.125688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.126261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.126618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.128737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.155414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 965/1938] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:03 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:54:03.159688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.173601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22f8a5-508d-4ef8-a025-c7038297ce71 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.178462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.179076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.179355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.179630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.180336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.182450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.195714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 974/1939] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:03.198021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.208051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba1e5-7ecb-4b80-953a-050e67274276 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.213315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.213910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.214194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.214466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.215139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.216739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad61c9-cb54-4c4d-b70e-95cbc3a3085c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.226557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 966/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:54:03.230934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.241114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d18f5-52f0-40be-8512-caab88e215af admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.246404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.247085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.247361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.247633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] PATH_INFO: `/v3/projects/35543544e0514fa6a1aef9499990ad9e/users/1be01840e97641dd96704c35a44d2a30/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.262471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35543544e0514fa6a1aef9499990ad9e, user_id=1be01840e97641dd96704c35a44d2a30, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.265123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.277975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 975/1941] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:03 2022] PUT /identity/v3/projects/35543544e0514fa6a1aef9499990ad9e/users/1be01840e97641dd96704c35a44d2a30/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:03.288795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.289081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.289385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.319196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] MFA Rules not processed for user `1be01840e97641dd96704c35a44d2a30`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:03.320432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:03.385092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 967/1942] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:03.389476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.449864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3cb42-581a-4933-acec-87610866ca17 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.454424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.455048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.455327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.455618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.456240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.458121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.485714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 976/1943] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:03 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:03.489858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.501100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ee0e7-bc1c-4f0c-8278-efa64977ca8e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.558253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.558903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.559190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.559465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.560105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.562036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.584281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 968/1944] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:03 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:03.588361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.598485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9fc99-0655-4339-a4ec-245488ce0d9a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.603171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.603816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.604099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.604370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.605015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.606621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.617109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 977/1945] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:03.621862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.631829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df66df-fba3-4448-ba6d-9299162982b0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.636965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.637545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.637820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.638087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] PATH_INFO: `/v3/projects/690f06c45dcb465292159b1f73de8bc0/users/2d452cc4b5454fad8cecc68d071e5450/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.652506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690f06c45dcb465292159b1f73de8bc0, user_id=2d452cc4b5454fad8cecc68d071e5450, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.655184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.668123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 969/1946] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:03 2022] PUT /identity/v3/projects/690f06c45dcb465292159b1f73de8bc0/users/2d452cc4b5454fad8cecc68d071e5450/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:03.671901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.682657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb4674-8f4b-4de0-b67a-94ed109000c9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.738194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.738834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.739111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.739379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.740023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.741581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428204d-7fd2-4b40-8dc8-cb8b0ea0a682 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.753577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 978/1947] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:03.757629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.765063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:03.769230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96484a-ad0e-43eb-8b32-0cb4fedf33b7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.774750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.775406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.775686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.775960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] PATH_INFO: `/v3/projects/690f06c45dcb465292159b1f73de8bc0/users/2d452cc4b5454fad8cecc68d071e5450/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.777873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690f06c45dcb465292159b1f73de8bc0, user_id=2d452cc4b5454fad8cecc68d071e5450, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:03.780552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:03.791185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 970/1948] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:03 2022] PUT /identity/v3/projects/690f06c45dcb465292159b1f73de8bc0/users/2d452cc4b5454fad8cecc68d071e5450/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:03.801351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.801635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.801910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:03.830271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] MFA Rules not processed for user `2d452cc4b5454fad8cecc68d071e5450`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:03.831514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:03.843135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:03.909144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 979/1949] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:03.921270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:03.923328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:03.991139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace716f-eed5-4dba-9826-93cec38bc646 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:03.995356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:03.996061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:03.996413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:03.996757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:04.007344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:04.008956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:04.020705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 971/1950] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:04.081980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:04.093297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d58fbb-66da-4f8a-9946-d417ed6dd8ae None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:04.097007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:04.097618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:04.097899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:04.098172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:04.164145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:04.165496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a02738-4206-4620-93c6-a909c7c2a4bc service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:04.174338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 980/1951] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:09.249995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:09.312104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccca910-8222-4e8f-828d-176927366fc6 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:09.317239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:09.318027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:09.318440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:09.319235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] PATH_INFO: `/v3/users/971c1d5226434eef816e2bd248fecd8b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:09.320599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=971c1d5226434eef816e2bd248fecd8b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:09.323677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:09.381858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971c1d5226434eef816e2bd248fecd8b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.393925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971c1d5226434eef816e2bd248fecd8b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.401621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971c1d5226434eef816e2bd248fecd8b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.408765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971c1d5226434eef816e2bd248fecd8b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.409410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971c1d5226434eef816e2bd248fecd8b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.415906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 972/1952] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:09 2022] DELETE /identity/v3/users/971c1d5226434eef816e2bd248fecd8b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:09.420297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:09.420777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:09.420875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:09.440028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:09.441255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:09.501487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 981/1953] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:09.511419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:09.512849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:09.527977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:09.572853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735fd3c-92bf-44c3-86cb-706618997372 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:09.576486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:09.577032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:09.577312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:09.577582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:09.586345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:09.588575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:09.597900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 973/1954] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:09.862666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:09.864837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:09.878596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13bb1-17f4-426e-b37c-af5130246a86 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:09.884552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:09.885275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:09.885687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:09.886010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] PATH_INFO: `/v3/projects/237073c227614875859f9b6eb4b6bce4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:09.887210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237073c227614875859f9b6eb4b6bce4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:09.889198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:09.936115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:09.941782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237073c227614875859f9b6eb4b6bce4', 'request_id': 'req-5fb5a0e3-4752-4a99-b37c-72e36f457827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.949290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237073c227614875859f9b6eb4b6bce4', 'request_id': 'req-5fb5a0e3-4752-4a99-b37c-72e36f457827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.958458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237073c227614875859f9b6eb4b6bce4', 'request_id': 'req-5fb5a0e3-4752-4a99-b37c-72e36f457827'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.975300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] The token cache is being invalidate because project 237073c227614875859f9b6eb4b6bce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:09.975760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.976504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:09.981284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 982/1955] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:09 2022] DELETE /identity/v3/projects/237073c227614875859f9b6eb4b6bce4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:09.982600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:09.995372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ed984-5c1f-4fdd-a4ae-633e0de83a0c service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:10.057940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:10.058724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:10.059072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:10.059410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] PATH_INFO: `/v3/users/1be01840e97641dd96704c35a44d2a30` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:10.060607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be01840e97641dd96704c35a44d2a30)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:10.062650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:10.117308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be01840e97641dd96704c35a44d2a30'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.130991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be01840e97641dd96704c35a44d2a30'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.139782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be01840e97641dd96704c35a44d2a30'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.147177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be01840e97641dd96704c35a44d2a30'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.147801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be01840e97641dd96704c35a44d2a30'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.153624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 974/1956] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:09 2022] DELETE /identity/v3/users/1be01840e97641dd96704c35a44d2a30 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:10.707873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:10.709380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:10.724722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5a0e3-4752-4a99-b37c-72e36f457827 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:10.784360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:10.784992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:10.785271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:10.785543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] PATH_INFO: `/v3/projects/35543544e0514fa6a1aef9499990ad9e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:10.786696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35543544e0514fa6a1aef9499990ad9e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:10.788589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:10.836716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35543544e0514fa6a1aef9499990ad9e', 'request_id': 'req-1e83efa2-f53d-4aea-95f8-972a69d3580c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.842307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35543544e0514fa6a1aef9499990ad9e', 'request_id': 'req-1e83efa2-f53d-4aea-95f8-972a69d3580c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.848419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35543544e0514fa6a1aef9499990ad9e', 'request_id': 'req-1e83efa2-f53d-4aea-95f8-972a69d3580c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.861164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] The token cache is being invalidate because project 35543544e0514fa6a1aef9499990ad9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:10.861447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.862056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:10.863597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 983/1957] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:10 2022] DELETE /identity/v3/projects/35543544e0514fa6a1aef9499990ad9e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:10.867517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:10.869655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:10.882114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068405-d102-48f3-a4fd-458415222b9c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:10.913435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:10.940481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:10.941113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:10.941394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:10.941668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] PATH_INFO: `/v3/users/2d452cc4b5454fad8cecc68d071e5450` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:10.942838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d452cc4b5454fad8cecc68d071e5450)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:10.944751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:10.990488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d452cc4b5454fad8cecc68d071e5450'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.002940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d452cc4b5454fad8cecc68d071e5450'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.010156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d452cc4b5454fad8cecc68d071e5450'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.017052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d452cc4b5454fad8cecc68d071e5450'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.017700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bdf866e0-59d8-4aa7-8b45-3c00c0517775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d452cc4b5454fad8cecc68d071e5450'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.023594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 975/1958] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:10 2022] DELETE /identity/v3/users/2d452cc4b5454fad8cecc68d071e5450 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:11.341542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.343742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:11.359153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83efa2-f53d-4aea-95f8-972a69d3580c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.422593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.423246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.423532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.423819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] PATH_INFO: `/v3/projects/690f06c45dcb465292159b1f73de8bc0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.425120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=690f06c45dcb465292159b1f73de8bc0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.427116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.429464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:11.481497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690f06c45dcb465292159b1f73de8bc0', 'request_id': 'req-15eb6609-7070-4f12-adfa-057227732649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.487406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690f06c45dcb465292159b1f73de8bc0', 'request_id': 'req-15eb6609-7070-4f12-adfa-057227732649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.494294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690f06c45dcb465292159b1f73de8bc0', 'request_id': 'req-15eb6609-7070-4f12-adfa-057227732649'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.507551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] The token cache is being invalidate because project 690f06c45dcb465292159b1f73de8bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:11.507838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.508458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-15eb6609-7070-4f12-adfa-057227732649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:11.510454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 984/1959] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:11 2022] DELETE /identity/v3/projects/690f06c45dcb465292159b1f73de8bc0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:11.516423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.516706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.516977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.539092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:11.540285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:11.607327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 976/1960] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:11.616653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.616983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.617257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.637567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:11.638787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:11.647506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 985/1961] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:11.652014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.661672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fe211-5d96-4d50-99d3-3a0d2740c1e6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.666760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.667358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.667634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.667899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.668542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:54:11.668839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.670428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.682890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 977/1962] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:54:11.687336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.697095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedbb91-3691-431a-82e0-5fc60f870db9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.701841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.702457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.702777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.703052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.703699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.705620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.737262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 986/1963] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:11 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:11.739291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.749392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d414-934d-4c35-8ea9-79d3221d7679 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.790245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:11.808160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.808793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.809069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.809336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.809987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.811935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.834172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 978/1964] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:11 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:11.839066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.849320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ae9a9-0c3d-43c6-a3ce-1fefa019ebe5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.853990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.854625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.854907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.855184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.855835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.857423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.867402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 987/1965] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:11.871891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.881800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdb6e3-2291-45e0-8807-80855d038008 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.887067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.887662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.887943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.888215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.888860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.890446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56b3da-6e84-4ca2-8395-66dcad567b9f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.900898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 979/1966] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:11.904957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:11.914946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0ddde-9319-4271-947b-dd35fc35f79a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:11.919671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:11.920345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.920631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.920914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] PATH_INFO: `/v3/projects/5fbd137bc40e4c55bc358c22bc228b26/users/fdd460f657874b878a18c65c0e9a6e57/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.935891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbd137bc40e4c55bc358c22bc228b26, user_id=fdd460f657874b878a18c65c0e9a6e57, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:11.938552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:11.952055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 988/1967] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:11 2022] PUT /identity/v3/projects/5fbd137bc40e4c55bc358c22bc228b26/users/fdd460f657874b878a18c65c0e9a6e57/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:11.962941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:11.963218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:11.963511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:11.990791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] MFA Rules not processed for user `fdd460f657874b878a18c65c0e9a6e57`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:11.991983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:12.055068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 980/1968] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:12.060143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.122559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd54-d3a1-40c4-9bb7-60d847a637fd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.127147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.127748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.128027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.128298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.128936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.130875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.157066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 989/1969] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:12 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:12.161252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.172604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e067fa-5918-4597-b48d-23a3abd4aff8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.226995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.227607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.228595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.228767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.229372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.231316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.255897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 981/1970] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:12 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:12.257068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.267954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8e15-82ec-4dff-b0b8-2fc9046941f5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.272544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.273150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.273435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.273709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.274355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.276939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.290395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 990/1971] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:12.292522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.302598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e75992-6d2f-4c7b-a5b2-9adba1ee53a2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.310249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.311961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.312244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.312516 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] PATH_INFO: `/v3/projects/b40537b5bf0647198a88599f27285ced/users/1508177dcbdc4fa6abe81f53819dcdb4/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.329742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40537b5bf0647198a88599f27285ced, user_id=1508177dcbdc4fa6abe81f53819dcdb4, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.332680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.349630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 982/1972] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:12 2022] PUT /identity/v3/projects/b40537b5bf0647198a88599f27285ced/users/1508177dcbdc4fa6abe81f53819dcdb4/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:54:12.353655 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.359617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:12.365119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703231-d7e9-403a-aa91-08a395852f9d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.429693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.430308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.430621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.430947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.431663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.433273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d25890-6dbb-4f20-965c-a895feae8788 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.443392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 991/1973] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:12.448382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.459489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4449dc4-94c6-4abb-8758-d95730f1e559 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.464937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.465595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.465892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.466151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] PATH_INFO: `/v3/projects/b40537b5bf0647198a88599f27285ced/users/1508177dcbdc4fa6abe81f53819dcdb4/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.468099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40537b5bf0647198a88599f27285ced, user_id=1508177dcbdc4fa6abe81f53819dcdb4, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.470763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.481773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 983/1974] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:12 2022] PUT /identity/v3/projects/b40537b5bf0647198a88599f27285ced/users/1508177dcbdc4fa6abe81f53819dcdb4/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:54:12.491707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.492079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.492417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.522341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] MFA Rules not processed for user `1508177dcbdc4fa6abe81f53819dcdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:12.527534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:12.625623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 992/1975] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:12.637379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.639179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:12.644488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:12.700667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463456a6-cee8-42dd-b501-03529479ff2f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.704283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.704895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.705175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.705450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:12.714912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:12.716339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:12.725617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 984/1976] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:12.943249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:12.953977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1f21d-2913-49de-8364-ad5f308f47e1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:12.957611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:12.958208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:12.958487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:12.958797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:13.015296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:13.016651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c539c9-2553-427b-903d-f67bc2ecf33d service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:13.025702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 993/1977] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:13.293365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:13.354966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eacf68-c43b-417f-abf6-cf0cd44411bf service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:13.359398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:13.360025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:13.360305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:13.360602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] PATH_INFO: `/v3/users/fdd460f657874b878a18c65c0e9a6e57` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:13.361710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd460f657874b878a18c65c0e9a6e57)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:13.363677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:13.410151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd460f657874b878a18c65c0e9a6e57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:13.422739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd460f657874b878a18c65c0e9a6e57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:13.432648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd460f657874b878a18c65c0e9a6e57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:13.441416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd460f657874b878a18c65c0e9a6e57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:13.442071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd460f657874b878a18c65c0e9a6e57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:13.448545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 985/1978] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:13 2022] DELETE /identity/v3/users/fdd460f657874b878a18c65c0e9a6e57 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:13.453004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:13.453296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:13.453579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:13.474013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:13.475265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:13.492718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:13.542888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 994/1979] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:13.553469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:13.555012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:13.612265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:13.650887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ed8dc-785f-420d-9d43-7f3e138ae14d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:13.656137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:13.657041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:13.657424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:13.657879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:13.670175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:13.671795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:13.687450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 986/1980] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:54:13.963906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:13.965529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:13.981148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09043-4428-4a87-acc9-541c1b867e3f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:13.992899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:13.994385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:13.994749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:13.995384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] PATH_INFO: `/v3/projects/5fbd137bc40e4c55bc358c22bc228b26` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:13.996422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbd137bc40e4c55bc358c22bc228b26)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:13.998345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:14.046643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbd137bc40e4c55bc358c22bc228b26', 'request_id': 'req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.052568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbd137bc40e4c55bc358c22bc228b26', 'request_id': 'req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.059316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbd137bc40e4c55bc358c22bc228b26', 'request_id': 'req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.073411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] The token cache is being invalidate because project 5fbd137bc40e4c55bc358c22bc228b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:14.073626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.074254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.075793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 995/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:13 2022] DELETE /identity/v3/projects/5fbd137bc40e4c55bc358c22bc228b26 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:14.079661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:14.090973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcabc9-40bd-4380-872b-0a10a42c10ca service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:14.153186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:14.153912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.154193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.154466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] PATH_INFO: `/v3/users/1508177dcbdc4fa6abe81f53819dcdb4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.155625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1508177dcbdc4fa6abe81f53819dcdb4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:14.157546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:14.202265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1508177dcbdc4fa6abe81f53819dcdb4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.214499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1508177dcbdc4fa6abe81f53819dcdb4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.221424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1508177dcbdc4fa6abe81f53819dcdb4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.228285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1508177dcbdc4fa6abe81f53819dcdb4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.228909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-63e0fd4c-bf7d-4d2f-a76b-718f0468ac3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1508177dcbdc4fa6abe81f53819dcdb4'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.234639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 987/1982] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:14 2022] DELETE /identity/v3/users/1508177dcbdc4fa6abe81f53819dcdb4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:14.503583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:14.505266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:14.527803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566cd3-9ad3-49e6-9038-a624edb0a5ed admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:14.584847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:14.585406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.585683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.585952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] PATH_INFO: `/v3/projects/b40537b5bf0647198a88599f27285ced` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.587144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40537b5bf0647198a88599f27285ced)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:14.589599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:14.645290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:14.647489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40537b5bf0647198a88599f27285ced', 'request_id': 'req-489971de-c602-4940-a11a-18af45bd33a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.655948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40537b5bf0647198a88599f27285ced', 'request_id': 'req-489971de-c602-4940-a11a-18af45bd33a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.663923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40537b5bf0647198a88599f27285ced', 'request_id': 'req-489971de-c602-4940-a11a-18af45bd33a2'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.677238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] The token cache is being invalidate because project b40537b5bf0647198a88599f27285ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:14.678298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.678298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-489971de-c602-4940-a11a-18af45bd33a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:14.679959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 996/1983] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:14 2022] DELETE /identity/v3/projects/b40537b5bf0647198a88599f27285ced => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:14.685555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.686024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.686117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.731887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:14.733903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:14.826792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:14.849262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 988/1984] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:54:14.858200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.858485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.858788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.883824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:14.886179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:14.896234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 997/1985] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:14.900029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:14.909899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d6f57-a2f4-4957-a329-8bb7e04382b9 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:14.914775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:14.916157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.916444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.916754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.917374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:54:14.917677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:14.919450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:14.932035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 989/1986] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:54:14.937856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:14.948351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cfa0d-f41d-44c3-8632-9a01db0e7ccc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:14.953607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:14.954501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:14.954888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:14.955241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:14.955955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:14.957927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:14.995362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 998/1987] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:14 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:14.999232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:15.014617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef9577-ac0b-44b7-9b38-8c5f59a971de admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:15.106551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:15.107196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.107477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.107748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.108391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:15.116020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:15.149057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 990/1988] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:14 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:54:15.154218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:15.166854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089e13-9d3b-4b23-8319-a0986736e6fb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:15.172294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:15.173072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.173425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.174551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.174788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:15.176191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:15.188447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 999/1989] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:15.192624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:15.202901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0eac3-f688-4758-9968-427a2a0045c5 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:15.208052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:15.208653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.208934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.209244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.209865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:15.211493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fff4cf-0ac2-4b31-a5d6-86eee5d3094e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:15.223851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 991/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:15.228064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:15.238496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e921f-70ca-4c75-a79d-9afc48c799a5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:15.243100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:15.243873 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.244158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.244447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] PATH_INFO: `/v3/projects/98b21ede9f52456e8055a86f51aa7c0f/users/d1f01524daae4f2f961005cad942b863/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.260152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b21ede9f52456e8055a86f51aa7c0f, user_id=d1f01524daae4f2f961005cad942b863, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:15.262923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:15.276613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1000/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:15 2022] PUT /identity/v3/projects/98b21ede9f52456e8055a86f51aa7c0f/users/d1f01524daae4f2f961005cad942b863/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:15.286906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.287259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.287640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.318970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] MFA Rules not processed for user `d1f01524daae4f2f961005cad942b863`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:15.320157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:15.383395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 992/1992] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:15.395123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:15.396758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:15.452679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64c972-7ffb-4fd9-9a05-f64ec9b70eee admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:15.456366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:15.456969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:15.457252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:15.457522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:15.466455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:15.467891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57b2b2-7f28-4ff8-98ed-f22cdc26f60d service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:15.476540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1001/1993] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:25.380540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:25.472677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:38.561469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:38.683273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bd7ee-e033-4bb7-93fe-9f2c4909c1a2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:38.687987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:38.688731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:38.689498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:38.689498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] PATH_INFO: `/v3/users/d1f01524daae4f2f961005cad942b863` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:38.691164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1f01524daae4f2f961005cad942b863)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:38.693102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:38.776476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1f01524daae4f2f961005cad942b863'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:38.798830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1f01524daae4f2f961005cad942b863'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:38.809738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1f01524daae4f2f961005cad942b863'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:38.816496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1f01524daae4f2f961005cad942b863'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:38.817138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1f01524daae4f2f961005cad942b863'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:38.823895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 993/1994] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:54:38 2022] DELETE /identity/v3/users/d1f01524daae4f2f961005cad942b863 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:38.828268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:38.828722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:38.828844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:38.849877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:38.851214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:38.916996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1002/1995] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:38.933418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:38.934894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:38.994278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40dae-c62b-4f61-9768-10b8cd1ba25d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:38.997968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:38.998599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:38.998880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:38.999148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.008441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.009893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922e125-3c64-4282-9897-f97f70acc81a service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.019352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 994/1996] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:39.371376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.372872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:39.387145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eaf4b6-edc1-4354-bb5b-11aebc31e752 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.391943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.392665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.393019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.393361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] PATH_INFO: `/v3/projects/98b21ede9f52456e8055a86f51aa7c0f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.394624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b21ede9f52456e8055a86f51aa7c0f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.396607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.448894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b21ede9f52456e8055a86f51aa7c0f', 'request_id': 'req-cc846cdb-60d7-4182-95a2-f3f7a48e732e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:39.454432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b21ede9f52456e8055a86f51aa7c0f', 'request_id': 'req-cc846cdb-60d7-4182-95a2-f3f7a48e732e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:39.460966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b21ede9f52456e8055a86f51aa7c0f', 'request_id': 'req-cc846cdb-60d7-4182-95a2-f3f7a48e732e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:39.473768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] The token cache is being invalidate because project 98b21ede9f52456e8055a86f51aa7c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:54:39.474059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:39.474750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-cc846cdb-60d7-4182-95a2-f3f7a48e732e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:54:39.476275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1003/1997] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:54:39 2022] DELETE /identity/v3/projects/98b21ede9f52456e8055a86f51aa7c0f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:39.482665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432351cc-a776-4406-9d57-66254266406c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.483030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432351cc-a776-4406-9d57-66254266406c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.483374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432351cc-a776-4406-9d57-66254266406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.503707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-432351cc-a776-4406-9d57-66254266406c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:39.504903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-432351cc-a776-4406-9d57-66254266406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:39.523433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:39.569068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 995/1998] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:54:39.578354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.578673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.578949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.599876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:39.601065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:39.609013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1004/1999] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:39.613122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432351cc-a776-4406-9d57-66254266406c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.623283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432351cc-a776-4406-9d57-66254266406c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.627887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.628481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.628762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.629033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.629673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:54:39.629974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.631590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.643566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 996/2000] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:54:39.647811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.657102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8be13-f434-40b1-8826-52a2c0ffeff9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.661863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.662458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.662773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.663044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.663683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.665776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.699131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1005/2001] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:54:39 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:54:39.703165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.722539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91772a7f-63a6-4e6f-b693-c9b4c1cd6d6c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.801314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.803111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.803577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.804020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.805402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.807722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.858282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 997/2002] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:54:39 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:54:39.862543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.880474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc3f5c-4a15-46b1-a547-eed1dfa0545a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.886485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.887133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.887407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.887675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.888482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.890082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.903139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1006/2003] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:54:39.907736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.920279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904b8d2-2931-4fec-921f-c2bcaff2d2aa admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.925725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.926561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.926771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.927040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.927694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.929291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f7e67-b03a-4e39-9b41-784f2d482764 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:39.942463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 998/2004] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:54:39.947996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:39.960558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69237e2-ccf4-4114-a842-a7575f9aec6d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:39.965791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:39.966445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:39.966760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:39.967038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] PATH_INFO: `/v3/projects/cf1d7bc0ada54427a1ed5fdd1bb7a0fb/users/ac91c08d1b554699bf7ddd8c3ae80d5c/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:39.985215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1d7bc0ada54427a1ed5fdd1bb7a0fb, user_id=ac91c08d1b554699bf7ddd8c3ae80d5c, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:39.988640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:40.007232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1007/2005] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:54:39 2022] PUT /identity/v3/projects/cf1d7bc0ada54427a1ed5fdd1bb7a0fb/users/ac91c08d1b554699bf7ddd8c3ae80d5c/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:54:40.017429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:40.017710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:40.017982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:40.046935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] MFA Rules not processed for user `ac91c08d1b554699bf7ddd8c3ae80d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:54:40.048177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:40.112732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 999/2006] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:54:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:54:40.125439 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:54:40.127152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:54:40.149122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:40.185273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57980c7-01f3-42d7-9fa1-3c82ef98a6c3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:54:40.188883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:54:40.189489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:54:40.189770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:54:40.190045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:54:40.198493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:54:40.199943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8826a6-49f9-49af-a498-cf190b2e2d74 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:54:40.208711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1008/2007] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:54:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:54:50.109600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:54:50.204675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=17 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:01.120173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:01.121632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:01.187667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c239cf-edbf-4986-9743-f827f5a94ced None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:01.192171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:01.192812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:01.193094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:01.193362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] PATH_INFO: `/v3/users/fecdd60f06af4878a56b19e572ab9086` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:01.194752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fecdd60f06af4878a56b19e572ab9086)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:01.196646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:01.245215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fecdd60f06af4878a56b19e572ab9086'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.258153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fecdd60f06af4878a56b19e572ab9086'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.266331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fecdd60f06af4878a56b19e572ab9086'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.273456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fecdd60f06af4878a56b19e572ab9086'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.274099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fecdd60f06af4878a56b19e572ab9086'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.280537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1000/2008] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:01 2022] DELETE /identity/v3/users/fecdd60f06af4878a56b19e572ab9086 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:01.285173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:01.285662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:01.285751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:01.332362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:01.334852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:01.426356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1009/2009] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:01.443709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:01.445435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:01.509344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524e74-261f-4712-ba0e-97d26aa25787 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:01.513835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:01.514530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:01.514830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:01.515122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:01.525103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:01.526565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:01.536438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1001/2010] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:01.814904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:01.816506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:01.845831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69845b-d6d4-4699-8e04-7a69a5ea65c2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:01.855408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:01.856601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:01.856878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:01.857197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] PATH_INFO: `/v3/projects/4187a5ed9e0a44aea442b8fb59baf2b5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:01.858325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4187a5ed9e0a44aea442b8fb59baf2b5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:01.860253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:01.913768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4187a5ed9e0a44aea442b8fb59baf2b5', 'request_id': 'req-caae91e4-49a5-4718-b88e-9f9c7536e0b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.920330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4187a5ed9e0a44aea442b8fb59baf2b5', 'request_id': 'req-caae91e4-49a5-4718-b88e-9f9c7536e0b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.926708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4187a5ed9e0a44aea442b8fb59baf2b5', 'request_id': 'req-caae91e4-49a5-4718-b88e-9f9c7536e0b8'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.939599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] The token cache is being invalidate because project 4187a5ed9e0a44aea442b8fb59baf2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:01.939889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.940512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:01.942232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1010/2011] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:01 2022] DELETE /identity/v3/projects/4187a5ed9e0a44aea442b8fb59baf2b5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:01.946536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:01.956299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537bbd9-5eab-4b91-973c-7abfc9efb854 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:01.976731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:02.022649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:02.023552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:02.023738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:02.024022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] PATH_INFO: `/v3/users/95c33ab80ea14b25b874cb61420dbc40` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:02.025715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c33ab80ea14b25b874cb61420dbc40)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:02.028114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:02.107665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c33ab80ea14b25b874cb61420dbc40'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.132083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c33ab80ea14b25b874cb61420dbc40'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.147616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c33ab80ea14b25b874cb61420dbc40'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.159579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c33ab80ea14b25b874cb61420dbc40'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.160366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c5ee3dc0-7bb2-4d19-b722-323792aeafd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c33ab80ea14b25b874cb61420dbc40'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.178326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1002/2012] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:01 2022] DELETE /identity/v3/users/95c33ab80ea14b25b874cb61420dbc40 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:02.660960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:02.662797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:02.676393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae91e4-49a5-4718-b88e-9f9c7536e0b8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:02.740641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:02.741322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:02.741602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:02.741840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] PATH_INFO: `/v3/projects/35b3209cb5094dac94fb9b7d759d975c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:02.743956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b3209cb5094dac94fb9b7d759d975c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:02.746978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:02.782085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:02.800548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b3209cb5094dac94fb9b7d759d975c', 'request_id': 'req-134afa5b-76f7-49c5-a227-0650e5dbad2b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.806391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b3209cb5094dac94fb9b7d759d975c', 'request_id': 'req-134afa5b-76f7-49c5-a227-0650e5dbad2b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.814318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b3209cb5094dac94fb9b7d759d975c', 'request_id': 'req-134afa5b-76f7-49c5-a227-0650e5dbad2b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.838235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] The token cache is being invalidate because project 35b3209cb5094dac94fb9b7d759d975c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:02.838546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.839207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-134afa5b-76f7-49c5-a227-0650e5dbad2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:02.843134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1011/2013] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:02 2022] DELETE /identity/v3/projects/35b3209cb5094dac94fb9b7d759d975c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:02.848103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:02.848580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:02.848863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:02.878988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:02.880137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:03.028940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1003/2014] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:03.037182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.037478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.037764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.062673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:03.063951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:03.073338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1012/2015] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:03.078272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.092951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a747c-929f-4c17-a945-b4dba3df6255 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.098627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.099284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.099561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.099832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.100582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:03.100928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.102774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.122556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1004/2016] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:03.126054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.138045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76f9c4-07c9-4954-9e1c-cbafaefaa4d0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.143962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.144707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.145046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.145386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.146703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.148214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.197106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1013/2017] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:03 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:03.199192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.211999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc3e2a-afaa-4bc4-a5cc-499b835bbac4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.249470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:03.296236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.296957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.298324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.298672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.299820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.301824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.328430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1005/2018] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:03 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:03.332636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.343180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca0cf-813e-42df-a5d9-e97d7b1e1042 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.350053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.350730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.351059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.351332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.351991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.353588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.370070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1014/2019] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:03.371998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.394011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d846fad-e74a-4b59-8ae2-0d1ed6a28e58 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.400439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.401065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.401343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.401614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.402318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.405291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c7a13-840a-4b63-b209-58f9ac909e1d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.426619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1006/2020] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:55:03.430697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.448756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45339c-7af7-4793-8fbc-7d812463e72a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.456447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.457506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.457795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.458072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] PATH_INFO: `/v3/projects/45bc02edb1044a9781b37120b695ff87/users/50ed6a22322c4086bc01fe99c4e3354f/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.506063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bc02edb1044a9781b37120b695ff87, user_id=50ed6a22322c4086bc01fe99c4e3354f, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.523369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.555456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1015/2021] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:03 2022] PUT /identity/v3/projects/45bc02edb1044a9781b37120b695ff87/users/50ed6a22322c4086bc01fe99c4e3354f/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:03.557619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.557908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.558187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.585935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:03.587186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:03.658721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1007/2022] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:03.671328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:03.674261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:03.747489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905b683-0b60-47b5-8fa9-ba92ae6b3d90 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:03.751297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:03.752037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:03.752421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:03.752793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:03.764574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:03.766140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f387-82bc-4018-ac55-515e71cd1cc4 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:03.778153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1016/2023] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:04.618998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:04.620957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:04.647281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a613a1-3967-4967-ae2f-5c310dd97349 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:04.652620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:04.653932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:04.654288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:04.655256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] PATH_INFO: `/v3/users/45aec9064c4e444a917f52d2f947461b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:04.656706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45aec9064c4e444a917f52d2f947461b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:04.659600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:04.761294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45aec9064c4e444a917f52d2f947461b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:04.836313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45aec9064c4e444a917f52d2f947461b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:04.862973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45aec9064c4e444a917f52d2f947461b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:04.904914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45aec9064c4e444a917f52d2f947461b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:04.907281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45aec9064c4e444a917f52d2f947461b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:04.928657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1008/2024] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:04 2022] DELETE /identity/v3/users/45aec9064c4e444a917f52d2f947461b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:04.933203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f365-d249-4947-b19f-738c6625ddb2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:04.935361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f365-d249-4947-b19f-738c6625ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:04.936002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f365-d249-4947-b19f-738c6625ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:04.987974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3156f365-d249-4947-b19f-738c6625ddb2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:04.989470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3156f365-d249-4947-b19f-738c6625ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:05.104961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:05.156709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1017/2025] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:05.180952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:05.182769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:05.220378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:05.325631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb59d04-b091-4f66-ac95-151bdf56898e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:05.350860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:05.354545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.354545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.354545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.407212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:05.409441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:05.435815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1009/2026] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:05.488029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.488269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.488520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.559062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] MFA Rules not processed for user `50ed6a22322c4086bc01fe99c4e3354f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:05.560703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:05.682016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1018/2027] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:05.687496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:05.689527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:05.710238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261547-d16f-4df0-a6f1-83215e3c151f service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:05.718108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:05.719313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.719743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.720213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.721026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:05.724262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:05.763925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1010/2028] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:05 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:05.767872 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:05.786111 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e84f6-1275-471b-8bc3-c51d1354d7dc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:05.891125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:05.891894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.892410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.892776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.893421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:05.895688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:05.921148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1019/2029] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:05 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:05.923433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:05.934747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c74c5d-b2a8-4dac-a0a3-b1ba751f4c87 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:05.941510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:05.942492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.942931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.943266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.944249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:05.946454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:05.958583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1011/2030] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:05.962857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:05.973781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35114f3d-7892-4684-ab83-4d5924a1da83 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:05.978539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:05.979203 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:05.979484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:05.979759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] PATH_INFO: `/v3/projects/e806ebf64e7b429ca183932174af6f9e/users/8acbef3dfb934c0c9a983e261d18d373/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:05.996939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e806ebf64e7b429ca183932174af6f9e, user_id=8acbef3dfb934c0c9a983e261d18d373, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:05.999798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.015863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.017747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:06.018210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1020/2031] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:05 2022] PUT /identity/v3/projects/e806ebf64e7b429ca183932174af6f9e/users/8acbef3dfb934c0c9a983e261d18d373/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:06.022009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.035226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e167e-3c75-41be-81c0-26ef7e7eca03 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.077045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:06.097308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.097683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702cfd9-3c0c-408f-8aa2-c63eb51e3b26 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.098108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.098463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.098958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] PATH_INFO: `/v3/projects/e270a066f14d43069b2d5867d01fb373` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.100070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=e270a066f14d43069b2d5867d01fb373)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.101986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.103009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.103695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.104326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.104326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.105038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.107587 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.123709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1021/2032] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:06.127864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.139996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3572-01eb-4428-9eae-db90af101985 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.145134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.146202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.146588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.146869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] PATH_INFO: `/v3/projects/e806ebf64e7b429ca183932174af6f9e/users/8acbef3dfb934c0c9a983e261d18d373/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.159851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e806ebf64e7b429ca183932174af6f9e, user_id=8acbef3dfb934c0c9a983e261d18d373, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.160720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e270a066f14d43069b2d5867d01fb373', 'request_id': 'req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:06.166539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd333f-193a-4fcc-b304-1f01de8b1d69 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.171030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e270a066f14d43069b2d5867d01fb373', 'request_id': 'req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:06.179746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e270a066f14d43069b2d5867d01fb373', 'request_id': 'req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:06.190552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1022/2033] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:06 2022] PUT /identity/v3/projects/e806ebf64e7b429ca183932174af6f9e/users/8acbef3dfb934c0c9a983e261d18d373/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:06.195791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] The token cache is being invalidate because project e270a066f14d43069b2d5867d01fb373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:06.196283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:06.197197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1f5c92a0-a01e-467c-a27a-b4d1aede0ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:06.198886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1012/2034] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:06 2022] DELETE /identity/v3/projects/e270a066f14d43069b2d5867d01fb373 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:55:06.203956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.204337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.204710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.241702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:06.243122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:06.314551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1023/2035] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:06.324391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.325258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.325685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.350483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:06.351874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:06.364272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1013/2036] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:06.368622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.381152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5523d-6627-4a4d-8287-c9b810bf65c6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.390506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.391029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.392031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.392317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.393205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:06.393583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.395339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.414558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1024/2037] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:06.417599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.428611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09628b72-e27b-451e-bacc-cacfaeae2e80 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.434686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.435310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.436122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.436439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.437342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.439462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.453340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:06.476547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1014/2038] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:06.480564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.492907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b3c7e-daf7-4947-9ae6-a5ef04aa82cf admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.576047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.576702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.576986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.577273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.577931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.580259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.620684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1025/2039] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:06 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:06.621755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.636803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8fd69-c872-4cde-b2c1-9986ca8a18f1 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.649686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.650383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.650690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.651173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.651835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.656043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.679458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1015/2040] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:06.684838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.702237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0460569-7480-4f37-b103-65c3fe208aa1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.717979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.719319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.719664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.720016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.721137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.723653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f426294-c52a-4d11-81a9-55378926ea44 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.750147 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1026/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:06.754789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.772924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27614a3c-64cd-4ea0-a549-e9b6d0e3f211 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:06.789320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:06.790349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.790849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.791269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] PATH_INFO: `/v3/projects/9a0a952730f841249662bacd558eda11/users/b228348894fb4aad86f9d58f59c49641/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.810929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0a952730f841249662bacd558eda11, user_id=b228348894fb4aad86f9d58f59c49641, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:06.813669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:06.833238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1016/2042] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:06 2022] PUT /identity/v3/projects/9a0a952730f841249662bacd558eda11/users/b228348894fb4aad86f9d58f59c49641/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:06.844565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01f82-19ac-40df-989f-fef52291e226 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:06.845355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01f82-19ac-40df-989f-fef52291e226 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:06.845640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01f82-19ac-40df-989f-fef52291e226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:06.889893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-58c01f82-19ac-40df-989f-fef52291e226 None None] MFA Rules not processed for user `b228348894fb4aad86f9d58f59c49641`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:06.891476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58c01f82-19ac-40df-989f-fef52291e226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:06.920131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:06.964278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1027/2043] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:06.975214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:06.977150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:07.032084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:07.032470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:07.032752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:07.043690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe05e1-f1ed-49a0-b10a-d1c89df88bfc admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:07.047510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:07.048200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:07.048485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:07.048757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:07.058148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:07.060319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:07.070010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1017/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:07.073465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] MFA Rules not processed for user `8acbef3dfb934c0c9a983e261d18d373`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:07.074768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:07.160010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1028/2045] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:07.172048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:07.181741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0c334-7e26-49ca-a892-3dd7f3149f6e service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:07.187805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:07.188533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:07.188677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:07.188948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:07.191190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:07.258757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:07.260337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:07.270613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1018/2046] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:09.775699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:09.777812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:09.790193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2371-0da7-4754-ad0a-d27b1d1516fa None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:09.794208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:09.794852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:09.795138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:09.795416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:09.805378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:09.806849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067c645-9d5f-4bc8-b2a7-15c23158a9df service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:09.817449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1029/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:17.267215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:19.813048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=55 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:23.731980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:23.733470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:23.798382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f4d1f-4a38-44fd-b461-dfc6f46ad5cb service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:23.802934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:23.803561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:23.803841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:23.804119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] PATH_INFO: `/v3/users/167cc6d2e68c49ada52ecc6764c832eb` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:23.805229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167cc6d2e68c49ada52ecc6764c832eb)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:23.807192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:23.851529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167cc6d2e68c49ada52ecc6764c832eb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:23.863889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167cc6d2e68c49ada52ecc6764c832eb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:23.871983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167cc6d2e68c49ada52ecc6764c832eb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:23.879127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167cc6d2e68c49ada52ecc6764c832eb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:23.879749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167cc6d2e68c49ada52ecc6764c832eb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:23.885709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1019/2048] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:23 2022] DELETE /identity/v3/users/167cc6d2e68c49ada52ecc6764c832eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:23.890716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:23.891193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:23.891303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:23.911131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:23.912374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:23.973783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1030/2049] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:23.984332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:23.985763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:24.047012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235537e-890b-4793-a6b4-835bf93ad7f2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:24.050657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:24.051299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:24.051578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:24.051846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:24.061519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:24.063002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:24.073223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1020/2050] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:24.374263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:24.376038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:24.392980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40a07e-1a2e-4f84-aca9-7bbb06ab11a2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:24.425241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:24.426200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:24.426578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:24.428309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] PATH_INFO: `/v3/projects/397c81e0065b4d2289720855ad8da5f7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:24.432103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] RBAC: Authorizing `identity:delete_project(project_id=397c81e0065b4d2289720855ad8da5f7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:24.433545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:24.511605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397c81e0065b4d2289720855ad8da5f7', 'request_id': 'req-c626256c-419b-45bc-a97c-f0ce8cf0165c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.524238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397c81e0065b4d2289720855ad8da5f7', 'request_id': 'req-c626256c-419b-45bc-a97c-f0ce8cf0165c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.536513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397c81e0065b4d2289720855ad8da5f7', 'request_id': 'req-c626256c-419b-45bc-a97c-f0ce8cf0165c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.563389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] The token cache is being invalidate because project 397c81e0065b4d2289720855ad8da5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:24.564086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.565817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.567816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1031/2051] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:24 2022] DELETE /identity/v3/projects/397c81e0065b4d2289720855ad8da5f7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:24.575240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:24.603755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef046-2e07-4763-875c-b4a9f2198547 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:24.628082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:24.684418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:24.685713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:24.686177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:24.686658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] PATH_INFO: `/v3/users/ec497855204e4e3cb82af745a8f52e41` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:24.687870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec497855204e4e3cb82af745a8f52e41)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:24.689835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:24.740847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec497855204e4e3cb82af745a8f52e41'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.753795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec497855204e4e3cb82af745a8f52e41'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.761196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec497855204e4e3cb82af745a8f52e41'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.770498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec497855204e4e3cb82af745a8f52e41'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.771443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a84b8cf8-61e9-4f2e-aa90-2650dfb1ed33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec497855204e4e3cb82af745a8f52e41'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:24.777333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1021/2052] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:24 2022] DELETE /identity/v3/users/ec497855204e4e3cb82af745a8f52e41 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:25.041664 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.043240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:25.056851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626256c-419b-45bc-a97c-f0ce8cf0165c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.115385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.116180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.116458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.116733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] PATH_INFO: `/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.117886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dd27a2e9b84e51926f7f2eb394c8e8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.119807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.150535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:25.167533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dd27a2e9b84e51926f7f2eb394c8e8', 'request_id': 'req-b0097f9a-0da2-4b0f-807c-117cafcf3020'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:25.173674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dd27a2e9b84e51926f7f2eb394c8e8', 'request_id': 'req-b0097f9a-0da2-4b0f-807c-117cafcf3020'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:25.179639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dd27a2e9b84e51926f7f2eb394c8e8', 'request_id': 'req-b0097f9a-0da2-4b0f-807c-117cafcf3020'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:25.192453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] The token cache is being invalidate because project 02dd27a2e9b84e51926f7f2eb394c8e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:25.192740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:25.193360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b0097f9a-0da2-4b0f-807c-117cafcf3020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:25.195124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1032/2053] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:25 2022] DELETE /identity/v3/projects/02dd27a2e9b84e51926f7f2eb394c8e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:25.200105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.200524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.200807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.221269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:25.222419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:25.280709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1022/2054] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:25.288394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.288675 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.288945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.309418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:25.310813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:25.319656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1033/2055] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:25.323571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.334707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5ef96-4f9a-4834-8343-ebb7dc6d35d0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.339331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.339936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.340217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.340489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.341648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:25.341952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.343647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.355311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1023/2056] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:25.360228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.369442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166361e2-981e-456a-b8fe-f4372bb30fa3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.374483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.375164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.375441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.375717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.376486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.378384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.406211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1034/2057] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:25.411114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.422166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9d6f-4407-4e73-a043-0f1c74f08b95 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.447669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:25.483340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.483965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.484243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.484518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.485156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.487117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.511542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1024/2058] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:55:25.512922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.523241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647cc02-4796-4acd-b01a-c731facd5090 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.527831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.528918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.529204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.529477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.530125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.531817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.541592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1035/2059] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:25.545460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.555502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0af45-a000-4ffc-9c43-374538f95083 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.560549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.561137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.561415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.561685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.562327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.564005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e24fd-713e-4f08-ad2e-013f50485099 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.576943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1025/2060] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:25.581131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.591756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12417677-5a91-42fa-bfb6-ecdb107dce68 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.596372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.597019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.597300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.597575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] PATH_INFO: `/v3/projects/6ba694390c8e4dc3b9b041358be54667/users/570d65c515484b43b8aedd59a3ea37a7/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.612979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba694390c8e4dc3b9b041358be54667, user_id=570d65c515484b43b8aedd59a3ea37a7, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.615703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.630690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1036/2061] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:25 2022] PUT /identity/v3/projects/6ba694390c8e4dc3b9b041358be54667/users/570d65c515484b43b8aedd59a3ea37a7/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:25.642867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.643157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.643428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.673752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] MFA Rules not processed for user `570d65c515484b43b8aedd59a3ea37a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:25.675072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:25.769464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1026/2062] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:25.773893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.848102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490e0d-cfd3-4d0f-acf6-89a08060b81b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.852772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.853405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.853689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.853960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.854627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.857045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.885574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1037/2063] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:25.890585 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.901534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23116926-a7b9-4333-94bb-6d5ed45c3bdb None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:25.958345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:25.958977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:25.959262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:25.959538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:25.960171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:25.962092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:25.983952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1027/2064] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:25 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:25.988073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:25.998429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236eab8-518b-4dc8-8f37-bacc2654ce5e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.003286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.003944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.004291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.004569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.005268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.006892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.017972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1038/2065] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:26.022116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:26.033516 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a349c8-2d3d-42de-945f-3c6fdc692e8c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.038392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.039091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.039379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.039654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] PATH_INFO: `/v3/projects/de740642d3d84717bd4069c687ad3bf3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.055211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de740642d3d84717bd4069c687ad3bf3, user_id=f6dccfb3d0d24ff2ae58aeb16107a5d9, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.057883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.071258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1028/2066] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:26 2022] PUT /identity/v3/projects/de740642d3d84717bd4069c687ad3bf3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:26.075910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:26.086445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532431ba-ce2b-454d-9a77-7054c79d78d9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.108524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:26.154412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.155063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.155345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.155620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.156265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.157857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37115f01-eef4-4d39-8f7e-54a7e8be2876 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.168207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1039/2067] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:55:26.173544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:26.179672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:26.187511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddb39e-af79-454c-a135-fe672582c3b6 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.193007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.194074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.194420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.194808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] PATH_INFO: `/v3/projects/de740642d3d84717bd4069c687ad3bf3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.196758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de740642d3d84717bd4069c687ad3bf3, user_id=f6dccfb3d0d24ff2ae58aeb16107a5d9, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.199636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.211435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1029/2068] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:26 2022] PUT /identity/v3/projects/de740642d3d84717bd4069c687ad3bf3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:26.222861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.223160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.223434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.254839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] MFA Rules not processed for user `f6dccfb3d0d24ff2ae58aeb16107a5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:26.256062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:26.326694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1040/2069] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:26.337945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:26.339668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:26.396686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c6860-4ba7-43b4-921f-7292daec6b9e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.400417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.401011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.401287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.401554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.410893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.412324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.422142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1030/2070] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:26.694015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:26.703988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218f8d5-76d3-49c2-8698-360efe213aa5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:26.707698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:26.708344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:26.708625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:26.708898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:26.767974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:26.769436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d9a66-51d8-4fa2-8df0-d6993e4232a6 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:26.779543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1041/2071] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:30.505526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:30.507019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:30.580673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb192-9c85-4a95-a12d-1e9ca1a92a16 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:30.585980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:30.586643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:30.586922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:30.587194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] PATH_INFO: `/v3/users/b228348894fb4aad86f9d58f59c49641` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:30.588338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b228348894fb4aad86f9d58f59c49641)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:30.590247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:30.646454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b228348894fb4aad86f9d58f59c49641'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:30.662341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b228348894fb4aad86f9d58f59c49641'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:30.676000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b228348894fb4aad86f9d58f59c49641'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:30.685434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b228348894fb4aad86f9d58f59c49641'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:30.686139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b228348894fb4aad86f9d58f59c49641'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:30.709639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1031/2072] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:30 2022] DELETE /identity/v3/users/b228348894fb4aad86f9d58f59c49641 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:30.714955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:30.715476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:30.715949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:30.751699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:30.753198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:30.836916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:30.873620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1042/2073] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:30.881031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:30.883068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:30.910857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:30.972308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5c6c6-c64a-4cfa-9658-bc64693b763c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:30.977351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:30.978112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:30.978530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:30.978978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:30.991291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:30.992779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a620f1-58c3-42dc-96a7-b96a5e1d5e9e service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.003418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1032/2074] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:31.320012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.322125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:31.337526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b107-9072-4d79-baa2-95ef48f4e17c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.343239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.344044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.344424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.344939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] PATH_INFO: `/v3/projects/9a0a952730f841249662bacd558eda11` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.346223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0a952730f841249662bacd558eda11)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.348271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.399653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0a952730f841249662bacd558eda11', 'request_id': 'req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:31.406101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0a952730f841249662bacd558eda11', 'request_id': 'req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:31.412496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0a952730f841249662bacd558eda11', 'request_id': 'req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:31.426327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] The token cache is being invalidate because project 9a0a952730f841249662bacd558eda11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:31.426644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:31.427278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bbc4f027-3259-4e6e-9e42-bea5d29cd5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:31.428799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1043/2075] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:31 2022] DELETE /identity/v3/projects/9a0a952730f841249662bacd558eda11 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:31.437113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.437113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.437113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.454258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:31.455491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:31.522658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1033/2076] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:31.528481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.528765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.529039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.548946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:31.550141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:31.557880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1044/2077] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:31.562625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.572359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc62d1-6238-43c3-a3cc-b2b99465e67b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.577010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.577601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.577879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.578148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.578827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:31.579128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.580707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.592034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1034/2078] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:31.596160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.605150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772caa7-0fce-4e5e-bff7-95377c80ed95 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.610401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.611022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.611296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.611565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.612197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.614076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.644952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1045/2079] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:31 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:31.646170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.655751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2a75b-50ac-4e5e-963b-6ba591ee196b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.718346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.718880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.719157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.719425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.720061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.722006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.745253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1035/2080] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:31 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:31.749633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.760153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07f5dc-2f47-46f6-be53-363e39c79c35 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.765426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.766155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.766497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.766915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.767461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.769149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.780427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1046/2081] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:31.784273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.802701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4f6e-432d-454a-bd46-ef32247a6976 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.807565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.808371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.808769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.809134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.810506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.812426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6989-671e-48b2-8333-98fa288e9e3b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.825349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1036/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:31.829562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.839706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba2e09-acd0-4c68-ad7b-508adac0bd87 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:31.844423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:31.845079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.845361 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.845635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] PATH_INFO: `/v3/projects/ad01763b3d76492598530234d3b75ec8/users/f4b71a4e5c1a4656a541f9cacec0af18/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.865384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad01763b3d76492598530234d3b75ec8, user_id=f4b71a4e5c1a4656a541f9cacec0af18, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:31.868008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:31.881927 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1047/2083] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:31 2022] PUT /identity/v3/projects/ad01763b3d76492598530234d3b75ec8/users/f4b71a4e5c1a4656a541f9cacec0af18/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:31.888524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18421ff-ae7c-4d5c-8c62-f49500a99ad6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:31.888524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18421ff-ae7c-4d5c-8c62-f49500a99ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:31.888524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18421ff-ae7c-4d5c-8c62-f49500a99ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:31.907362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c18421ff-ae7c-4d5c-8c62-f49500a99ad6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:31.908558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c18421ff-ae7c-4d5c-8c62-f49500a99ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:31.921597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:31.971421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1037/2084] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:31.980971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:31.983567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:32.042645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e5f01-96f9-4256-974e-7bf00d0f3031 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:32.046720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:32.047332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:32.047612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:32.047883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:32.057047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:32.058467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:32.069116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1048/2085] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:33.391845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9739cf4-c512-4db1-ba67-bea641ee42e7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:33.392385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9739cf4-c512-4db1-ba67-bea641ee42e7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:33.392385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9739cf4-c512-4db1-ba67-bea641ee42e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:33.433770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d9739cf4-c512-4db1-ba67-bea641ee42e7 None None] MFA Rules not processed for user `f4b71a4e5c1a4656a541f9cacec0af18`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:33.436219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d9739cf4-c512-4db1-ba67-bea641ee42e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:33.524201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1038/2086] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:33.538966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:33.542905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:33.560384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:33.641508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9789435-b205-4ffd-9add-81f38dce353b service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:33.648412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:33.649306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:33.649688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:33.650060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:33.660884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:33.662584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789941df-bc9b-426b-8ecb-c9e734f84568 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:33.685376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1049/2087] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:35.822954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:35.823844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:35.824151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:35.826809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1039/2088] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:55:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:55:35.860424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:35.861256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:35.861548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:35.885869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:35.888129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:35.926985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:35.928861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:35.946338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff6a8e-c99f-4171-b869-f687fddc49ea None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:35.951110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:35.951806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:35.952158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:35.952573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] PATH_INFO: `/v3/users/23126c9640d6433e96d393b885f3f829` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:35.953767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23126c9640d6433e96d393b885f3f829)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:35.955800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:35.970672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1050/2089] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:35.980629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:35.982799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:35.996110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6ec41-0d21-4712-865a-7eab2be6f17b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.002135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.002775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.003057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.003333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.004176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23126c9640d6433e96d393b885f3f829'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.011929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.014057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.016966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23126c9640d6433e96d393b885f3f829'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.024206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1051/2090] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:36.024941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23126c9640d6433e96d393b885f3f829'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.032029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23126c9640d6433e96d393b885f3f829'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.032699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-324e415b-c060-4ee4-ae85-1b2a1918a591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23126c9640d6433e96d393b885f3f829'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.040650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1040/2091] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:35 2022] DELETE /identity/v3/users/23126c9640d6433e96d393b885f3f829 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:36.223602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.237152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9cc17-cc06-4f11-9bfd-3e6bae767028 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.296256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.296887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.297170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.297447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] PATH_INFO: `/v3/projects/b68484aca102406a914066e2cc11a452` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.298604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68484aca102406a914066e2cc11a452)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.300918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.337849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:36.349676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68484aca102406a914066e2cc11a452', 'request_id': 'req-7b64e9e5-8823-4114-90d8-02ea6f115276'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.355220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68484aca102406a914066e2cc11a452', 'request_id': 'req-7b64e9e5-8823-4114-90d8-02ea6f115276'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.361225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68484aca102406a914066e2cc11a452', 'request_id': 'req-7b64e9e5-8823-4114-90d8-02ea6f115276'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.373982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] The token cache is being invalidate because project b68484aca102406a914066e2cc11a452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:36.374274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.374933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7b64e9e5-8823-4114-90d8-02ea6f115276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:36.376402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1052/2092] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:36 2022] DELETE /identity/v3/projects/b68484aca102406a914066e2cc11a452 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:36.381442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.381732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.382033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.402106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:36.404414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:36.421432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:36.465182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1041/2093] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:36.473192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.473442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.473716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.493356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:36.494580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:36.502936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1053/2094] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:36.507228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.517674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfb5fe-85de-4a53-bedd-1db8699a6643 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.522234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.522864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.523148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.523422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.524066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:36.524369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.525970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.537355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1042/2095] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:36.541527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.550984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb62ef-fc3f-459e-85ef-23f4a5c1d106 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.555700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.556302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.556579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.556849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.557479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.560107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.590913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1054/2096] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:36 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:36.592117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.602008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9de86-e034-42a5-94d4-f5d4b8e7eaf0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.659601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.660206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.660487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.660759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.661376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.663355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.684094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1043/2097] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:36 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:36.688715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.699308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92beec9f-45ff-4b0d-a72f-a0396384ee98 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.704622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.705224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.705505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.705781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.706426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.708094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.718084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1055/2098] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:36.721805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.731908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a9ef4-5489-4759-ac06-047e595483bd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.736516 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.737059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.737338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.737609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.738253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.739869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceb1f8-b091-4284-9107-bd741751fb35 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.755256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1044/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:36.757948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.769191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867d31-d570-47de-b25a-531bd907492a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.774849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.775542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.775789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.776199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] PATH_INFO: `/v3/projects/6e8a7ef82545435fa36586761e3c85c3/users/16e5bc69062549e5bebb5b33e1be492e/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.791062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8a7ef82545435fa36586761e3c85c3, user_id=16e5bc69062549e5bebb5b33e1be492e, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.793709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:36.808700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1056/2100] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:36 2022] PUT /identity/v3/projects/6e8a7ef82545435fa36586761e3c85c3/users/16e5bc69062549e5bebb5b33e1be492e/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:36.814366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25511f-0e41-4daf-bff4-faa2d72dfbeb None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.814822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25511f-0e41-4daf-bff4-faa2d72dfbeb None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.815163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25511f-0e41-4daf-bff4-faa2d72dfbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.835100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2a25511f-0e41-4daf-bff4-faa2d72dfbeb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:36.836338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2a25511f-0e41-4daf-bff4-faa2d72dfbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:36.903951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1045/2101] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:36.913360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:36.915198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:36.978717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2226a-5e8f-43f2-818b-0b6469d4ac0f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:36.982983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:36.983551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:36.983829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:36.984099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:36.993756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:36.995213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:37.004898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1057/2102] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:41.477752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cb30-30c3-478e-8f72-3a436bd92e79 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:41.478367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cb30-30c3-478e-8f72-3a436bd92e79 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:41.478367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cb30-30c3-478e-8f72-3a436bd92e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:41.511596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-be13cb30-30c3-478e-8f72-3a436bd92e79 None None] MFA Rules not processed for user `16e5bc69062549e5bebb5b33e1be492e`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:41.512867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-be13cb30-30c3-478e-8f72-3a436bd92e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:41.530435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:41.584220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1046/2103] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:41.598260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:41.599814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:41.662083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:41.679022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a58cd-8c1f-48d6-b45d-c0db622e2c8d service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:41.685486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:41.686189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:41.686470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:41.686776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:41.695517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:41.697204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571f60-baf7-42e7-a0e3-635efe7f7898 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:41.707424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1058/2104] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:42.536005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:42.537005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:42.537527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:42.558116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] MFA Rules not processed for user `f4b71a4e5c1a4656a541f9cacec0af18`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:42.559961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:42.623869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1047/2105] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:43.402311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85b4c-31e0-48ab-abf6-ac04efac88f8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:43.402869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85b4c-31e0-48ab-abf6-ac04efac88f8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:43.402869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85b4c-31e0-48ab-abf6-ac04efac88f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:43.426466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d0c85b4c-31e0-48ab-abf6-ac04efac88f8 None None] MFA Rules not processed for user `16e5bc69062549e5bebb5b33e1be492e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:43.427975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d0c85b4c-31e0-48ab-abf6-ac04efac88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:43.438471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1059/2106] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:47.011648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:47.013948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:47.047867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1c723-bcc1-4625-b591-43dfbeb074d4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:47.064879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:47.066274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:47.066750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:47.067256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] PATH_INFO: `/v3/users/570d65c515484b43b8aedd59a3ea37a7` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:47.068963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=570d65c515484b43b8aedd59a3ea37a7)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:47.071655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:47.182042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570d65c515484b43b8aedd59a3ea37a7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:47.197790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570d65c515484b43b8aedd59a3ea37a7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:47.210454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570d65c515484b43b8aedd59a3ea37a7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:47.221331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570d65c515484b43b8aedd59a3ea37a7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:47.222017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570d65c515484b43b8aedd59a3ea37a7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:47.230817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1048/2107] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:47 2022] DELETE /identity/v3/users/570d65c515484b43b8aedd59a3ea37a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:47.236041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838f42-d3c8-4aa3-a1e8-91327db37d1b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:47.236875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838f42-d3c8-4aa3-a1e8-91327db37d1b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:47.237385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838f42-d3c8-4aa3-a1e8-91327db37d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:47.267263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f7838f42-d3c8-4aa3-a1e8-91327db37d1b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:47.268504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f7838f42-d3c8-4aa3-a1e8-91327db37d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:47.344075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1060/2108] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:47.354176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:47.355897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:47.399171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:47.420722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593749b-33a1-4004-950f-5cf23a41a5ef admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:47.424910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:47.425530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:47.425807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:47.426077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:47.435605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:47.437033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd6b4-5469-4151-adab-13da333ad029 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:47.447423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1049/2109] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:47.974312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:47.974862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:47.975298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:47.978385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1061/2110] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:55:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:55:48.015658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:48.016655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:48.017620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:48.041565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] MFA Rules not processed for user `27b96a89a0d140aeaeba8078a21b4cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:48.043218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:48.135810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1050/2111] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:55:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:48.160004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:48.162817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:48.183561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9d2a0-bbac-4f67-b28c-4aaf54340384 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:48.190109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:48.191149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:48.191667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:48.192144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:48.217109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:48.220154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:48.241371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1062/2112] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:48.322862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:48.327787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:48.361714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1eb8a-3efe-4e15-9d28-48ed14a8527a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:48.384192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:48.385175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:48.385794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:48.386225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] PATH_INFO: `/v3/projects/6ba694390c8e4dc3b9b041358be54667` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:48.389132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba694390c8e4dc3b9b041358be54667)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:48.393129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:48.492562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba694390c8e4dc3b9b041358be54667', 'request_id': 'req-74088130-c630-450f-8dab-831f44579f57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.505000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba694390c8e4dc3b9b041358be54667', 'request_id': 'req-74088130-c630-450f-8dab-831f44579f57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.515083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba694390c8e4dc3b9b041358be54667', 'request_id': 'req-74088130-c630-450f-8dab-831f44579f57'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.530218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] The token cache is being invalidate because project 6ba694390c8e4dc3b9b041358be54667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:48.530487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.531249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.535432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1051/2113] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:48 2022] DELETE /identity/v3/projects/6ba694390c8e4dc3b9b041358be54667 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:48.539285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:48.557958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b47ca-bc97-474c-a8a0-4ad9ce4ff0ed service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:48.658384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:48.659430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:48.659999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:48.660472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] PATH_INFO: `/v3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:48.662186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6dccfb3d0d24ff2ae58aeb16107a5d9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:48.664914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:48.737826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6dccfb3d0d24ff2ae58aeb16107a5d9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.747528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6dccfb3d0d24ff2ae58aeb16107a5d9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.763937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:48.767407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6dccfb3d0d24ff2ae58aeb16107a5d9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.769153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6dccfb3d0d24ff2ae58aeb16107a5d9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.781791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6dccfb3d0d24ff2ae58aeb16107a5d9'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:48.795876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1063/2114] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:48 2022] DELETE /identity/v3/users/f6dccfb3d0d24ff2ae58aeb16107a5d9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:49.131686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:49.133880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.156286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74088130-c630-450f-8dab-831f44579f57 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:49.240967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:49.246474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:49.248392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:49.249087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:49.249617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] PATH_INFO: `/v3/users/ac91c08d1b554699bf7ddd8c3ae80d5c` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:49.251359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac91c08d1b554699bf7ddd8c3ae80d5c)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:49.253883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:49.308614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:49.312204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.329484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97579ce1-59ad-474b-b6c6-338f39453536 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:49.338660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:49.340021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:49.340667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:49.341398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] PATH_INFO: `/v3/users/50ed6a22322c4086bc01fe99c4e3354f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:49.343310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ed6a22322c4086bc01fe99c4e3354f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:49.346524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:49.369146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac91c08d1b554699bf7ddd8c3ae80d5c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.406237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac91c08d1b554699bf7ddd8c3ae80d5c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.421489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac91c08d1b554699bf7ddd8c3ae80d5c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.439828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac91c08d1b554699bf7ddd8c3ae80d5c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.440946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e66e6477-4d88-4948-9132-1fce0eaf2c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac91c08d1b554699bf7ddd8c3ae80d5c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.444877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ed6a22322c4086bc01fe99c4e3354f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.462383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ed6a22322c4086bc01fe99c4e3354f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.466696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1052/2115] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:49 2022] DELETE /identity/v3/users/ac91c08d1b554699bf7ddd8c3ae80d5c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:49.468262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:49.469185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:49.469980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:49.494547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ed6a22322c4086bc01fe99c4e3354f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.495741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ed6a22322c4086bc01fe99c4e3354f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.515046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ed6a22322c4086bc01fe99c4e3354f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.518242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:49.519524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.529593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1064/2116] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:49 2022] DELETE /identity/v3/users/50ed6a22322c4086bc01fe99c4e3354f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:49.544830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:49.546778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.706170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e59ae-5f3e-46ef-9a0e-085a2af28201 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:49.710025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1053/2117] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:49.721820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:49.722528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:49.723505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:49.723820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] PATH_INFO: `/v3/projects/de740642d3d84717bd4069c687ad3bf3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:49.727470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de740642d3d84717bd4069c687ad3bf3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:49.729905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:49.771943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:49.773775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.829812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de740642d3d84717bd4069c687ad3bf3', 'request_id': 'req-d5ceae83-0392-49f3-b299-f6e98cab306f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.842022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de740642d3d84717bd4069c687ad3bf3', 'request_id': 'req-d5ceae83-0392-49f3-b299-f6e98cab306f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.853044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:49.853044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de740642d3d84717bd4069c687ad3bf3', 'request_id': 'req-d5ceae83-0392-49f3-b299-f6e98cab306f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.876044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] The token cache is being invalidate because project de740642d3d84717bd4069c687ad3bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:49.876577 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.877353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d5ceae83-0392-49f3-b299-f6e98cab306f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:49.880458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1065/2118] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:49 2022] DELETE /identity/v3/projects/de740642d3d84717bd4069c687ad3bf3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:49.885660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533116c-1b9e-404f-9b03-42d69fece2ce None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:49.885915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533116c-1b9e-404f-9b03-42d69fece2ce None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:49.886333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533116c-1b9e-404f-9b03-42d69fece2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:49.930553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f60426-2253-4c25-a2c2-617cbf47a739 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:49.941027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8533116c-1b9e-404f-9b03-42d69fece2ce None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:49.944961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8533116c-1b9e-404f-9b03-42d69fece2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:49.996138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:50.047386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.048212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.048332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.048552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.052529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.096268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1066/2119] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:50.104608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.104934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.105224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.135157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:50.138089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.149414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1067/2120] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:50.155429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.168387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175a9bb-b16f-406c-98b3-a2b1a2e961a3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.175993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.176797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.177148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.177519 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.178931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:50.179500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.182332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.200707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.203318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.204636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1068/2121] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:50.207539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.212983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.220081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1054/2122] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:50.223565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.230419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5c237-d507-4a29-9754-1d42ac2593c7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.235344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcafae-67a9-491f-9e7f-00e5d27392b0 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.236657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.237683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.238148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.238626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] PATH_INFO: `/v3/projects/45bc02edb1044a9781b37120b695ff87` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.239990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45bc02edb1044a9781b37120b695ff87)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.242240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.242501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.243426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.243918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.244400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.245291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.248663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.303991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1055/2123] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:50 2022] POST /identity/v3/projects => generated 359 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:50.310690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.330382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45bc02edb1044a9781b37120b695ff87', 'request_id': 'req-f3c2f4bd-c6ee-4701-8548-4a4c09115212'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.336982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45bc02edb1044a9781b37120b695ff87', 'request_id': 'req-f3c2f4bd-c6ee-4701-8548-4a4c09115212'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.345232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45bc02edb1044a9781b37120b695ff87', 'request_id': 'req-f3c2f4bd-c6ee-4701-8548-4a4c09115212'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.377175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] The token cache is being invalidate because project 45bc02edb1044a9781b37120b695ff87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:50.377493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.378990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.381615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1069/2124] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:50 2022] DELETE /identity/v3/projects/45bc02edb1044a9781b37120b695ff87 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:50.385300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.386851 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.423769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76a205-3db7-4061-b1b7-8736c6c8ac3c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.505168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:50.527851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.528525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2f4bd-c6ee-4701-8548-4a4c09115212 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.529408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.529408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.529408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.530134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.532775 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.538248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.539285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.539810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.540392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] PATH_INFO: `/v3/users/8acbef3dfb934c0c9a983e261d18d373` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.542476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8acbef3dfb934c0c9a983e261d18d373)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.547569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.556349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:50.573674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1056/2125] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:50 2022] POST /identity/v3/users => generated 280 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:50.575324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.577692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.595121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de28ea-3723-4d31-9671-a6a4a76ff62e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.608610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.611683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.611967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.614882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.615263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.617926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.618678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8acbef3dfb934c0c9a983e261d18d373'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.632330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8acbef3dfb934c0c9a983e261d18d373'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.637081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1057/2126] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:50.641248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.656719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8acbef3dfb934c0c9a983e261d18d373'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.657635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8acbef3dfb934c0c9a983e261d18d373'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.660338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af79be-30c2-435c-8dc5-5d196426aac7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.667032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.668406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.668754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8acbef3dfb934c0c9a983e261d18d373'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:50.669221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.669700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.670710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.673057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5b896-f7fa-48a7-8073-3c59d3dadf3c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.685194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1070/2127] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:50 2022] DELETE /identity/v3/users/8acbef3dfb934c0c9a983e261d18d373 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:55:50.693622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1058/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:55:50.697963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:50.699616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:50.804693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adebfb-50ee-4c8f-8812-287d13c38bfb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:50.810810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:50.811684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.811937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.812288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] PATH_INFO: `/v3/projects/0bd41a96038443bc895d0b5d0ccf85ca/users/8a8bfa3ebf5d432994f3d2ad4b18557e/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.829620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd41a96038443bc895d0b5d0ccf85ca, user_id=8a8bfa3ebf5d432994f3d2ad4b18557e, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:50.832515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:50.852326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1071/2129] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:50 2022] PUT /identity/v3/projects/0bd41a96038443bc895d0b5d0ccf85ca/users/8a8bfa3ebf5d432994f3d2ad4b18557e/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:50.865960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:50.867704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:50.868007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:50.924340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] MFA Rules not processed for user `8a8bfa3ebf5d432994f3d2ad4b18557e`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:50.926997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.013040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.015862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.063870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1059/2130] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.130213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.136419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.201082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:51.235256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ad5e-aa7b-4074-9cff-6a7012bbee96 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.248425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.249105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.249393 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.249670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] PATH_INFO: `/v3/projects/cf1d7bc0ada54427a1ed5fdd1bb7a0fb` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.256670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1d7bc0ada54427a1ed5fdd1bb7a0fb)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.259877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.264356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b291d8c-4c8a-49d5-b2d7-f71d92a6cf37 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.275890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.285550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.291694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.292719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.317000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.320887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.346744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1d7bc0ada54427a1ed5fdd1bb7a0fb', 'request_id': 'req-d35284d5-309f-4c55-944a-f2a5e8bcd705'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.366090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1060/2131] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:51.371752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.373467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1d7bc0ada54427a1ed5fdd1bb7a0fb', 'request_id': 'req-d35284d5-309f-4c55-944a-f2a5e8bcd705'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.381636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1d7bc0ada54427a1ed5fdd1bb7a0fb', 'request_id': 'req-d35284d5-309f-4c55-944a-f2a5e8bcd705'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.389081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.430064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47f24e-d29b-4946-b7eb-faa1d44124bf service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.436469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] The token cache is being invalidate because project cf1d7bc0ada54427a1ed5fdd1bb7a0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:51.437363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.438035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.442058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1072/2132] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:51 2022] DELETE /identity/v3/projects/cf1d7bc0ada54427a1ed5fdd1bb7a0fb => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:51.442867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.447719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.510842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.511797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.512991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.512991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] PATH_INFO: `/v3/projects/e806ebf64e7b429ca183932174af6f9e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.515102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e806ebf64e7b429ca183932174af6f9e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.517064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.529488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:51.556492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35284d5-309f-4c55-944a-f2a5e8bcd705 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.567438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.570591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.571145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.571898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] PATH_INFO: `/v3/users/8a8bfa3ebf5d432994f3d2ad4b18557e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.573527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8bfa3ebf5d432994f3d2ad4b18557e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.576665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.585356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e806ebf64e7b429ca183932174af6f9e', 'request_id': 'req-fd6230ab-5b67-4562-b47f-865eade9cf64'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.598840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e806ebf64e7b429ca183932174af6f9e', 'request_id': 'req-fd6230ab-5b67-4562-b47f-865eade9cf64'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.609038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e806ebf64e7b429ca183932174af6f9e', 'request_id': 'req-fd6230ab-5b67-4562-b47f-865eade9cf64'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.630178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] The token cache is being invalidate because project e806ebf64e7b429ca183932174af6f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:51.630551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.631276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fd6230ab-5b67-4562-b47f-865eade9cf64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.633637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1061/2133] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:51 2022] DELETE /identity/v3/projects/e806ebf64e7b429ca183932174af6f9e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:51.635034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee050-fae4-4035-8bb3-c70ccb45226f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.635315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee050-fae4-4035-8bb3-c70ccb45226f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.635587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee050-fae4-4035-8bb3-c70ccb45226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.649098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8bfa3ebf5d432994f3d2ad4b18557e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.660220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ccbee050-fae4-4035-8bb3-c70ccb45226f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:51.660343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8bfa3ebf5d432994f3d2ad4b18557e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.661282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ccbee050-fae4-4035-8bb3-c70ccb45226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.685946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8bfa3ebf5d432994f3d2ad4b18557e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.686859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8bfa3ebf5d432994f3d2ad4b18557e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.696634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-8dfd1c42-c14f-451d-b727-2da809f24faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8bfa3ebf5d432994f3d2ad4b18557e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:51.716603 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1073/2134] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:55:51 2022] DELETE /identity/v3/users/8a8bfa3ebf5d432994f3d2ad4b18557e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:51.732863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034e7bd-6f33-4e38-a80a-b8d470029114 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.733288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034e7bd-6f33-4e38-a80a-b8d470029114 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.733575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034e7bd-6f33-4e38-a80a-b8d470029114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.747312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:51.749967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1062/2135] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.760366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.761391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.761673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.767191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c034e7bd-6f33-4e38-a80a-b8d470029114 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:51.768511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c034e7bd-6f33-4e38-a80a-b8d470029114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.816950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:51.818349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.865513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1074/2136] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.865513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7627-bdb2-45a6-96be-0d5c621a09a9 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.865513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7627-bdb2-45a6-96be-0d5c621a09a9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.865513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7627-bdb2-45a6-96be-0d5c621a09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.884339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d00c7627-bdb2-45a6-96be-0d5c621a09a9 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:51.886366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d00c7627-bdb2-45a6-96be-0d5c621a09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.901268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1075/2137] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.901268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.901268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.901268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.932706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:51.934738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1063/2138] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.935278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:51.935834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.950182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1076/2139] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:51.951520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:51.951762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c93d0a-89d5-4921-aa85-2cd8563e23ac None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.957599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.958790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.959140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.959483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.960386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:51.960805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.962735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.965813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae0807-4f31-429a-bcde-7debb1df7903 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:51.971688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:51.972805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:51.973298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:51.973966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:51.975267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:51.975802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:51.977703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:51.983972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1064/2140] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:51.985602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.003590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1077/2141] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:52.005962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.007046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6e825-54da-4130-a7d2-e7e4af563dd4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.013348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.015100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.015938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.016748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.017671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.018820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c1c4d-48d1-4d83-9382-5bbe54065820 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.020474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.025798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.026440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.026751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.027025 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.028059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.030997 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.068906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1065/2142] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:51 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:52.070892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.078404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:52.082574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1078/2143] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:52.087751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.189138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4daa6-6700-434c-9c79-906e6c89516c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.208683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9be9fc-5e0e-491e-b9ab-6fccb26e4248 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.216424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.217046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.217323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.217591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.218223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.221050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.258004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1079/2144] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:52.258004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.258226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.284049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.285060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.285345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.285612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.286238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.289444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.325649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1066/2145] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:52.325649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.334462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42e3a2-63ec-44b9-9ef7-43d4c93588c3 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.338053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:52.339562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.340745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.341030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.341304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.342092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.343832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.345180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d89af7-0b16-47e5-81c2-e53b9204b1cb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.351363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.353016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.353628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.354285 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.357031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1067/2146] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:52.358322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.370739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.371259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83d31-22d4-45c0-8092-c9794699fcce admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.373357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.378100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.379026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.379359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.379824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.380574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.382449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.392781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1080/2147] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:52.393472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.394560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1068/2148] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:52.398341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.404406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a844e-3786-4143-9a98-be27094f8774 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.409370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ffcb-27df-456f-a48b-99f860fe504c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.410929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.411535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.411807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.412074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.412992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.414711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.415125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.415889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.416175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.416539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.417278 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.418946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.427637 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1081/2149] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:52.430557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1069/2150] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:52.436180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.437299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.447639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622598c-07f8-4ff4-9b0a-99d609739da6 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.452848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ff727-9c47-4679-bea7-a2aac42803b9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.453666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.454340 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.454666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.454990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] PATH_INFO: `/v3/projects/6d05fbbec08d43468068ce162f427eeb/users/fb4bb9cd41cd46ddb87a746e732a948f/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.460094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.460776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.461316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.461427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] PATH_INFO: `/v3/projects/d01561573a044c87a4102522b8fe8432/users/5167ece674de4c11b71bb63db58f9eb3/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.471633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d05fbbec08d43468068ce162f427eeb, user_id=fb4bb9cd41cd46ddb87a746e732a948f, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.475178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae35783-cb37-49ca-9695-18ca384c7050 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.479865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01561573a044c87a4102522b8fe8432, user_id=5167ece674de4c11b71bb63db58f9eb3, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.483058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d30d21-b738-4614-a6f4-a8b1273cdcfd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.493522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1070/2151] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:52 2022] PUT /identity/v3/projects/6d05fbbec08d43468068ce162f427eeb/users/fb4bb9cd41cd46ddb87a746e732a948f/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:52.499570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d243ff-8039-445e-87bd-32e464510876 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.499934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d243ff-8039-445e-87bd-32e464510876 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.500210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d243ff-8039-445e-87bd-32e464510876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.502238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1082/2152] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:52 2022] PUT /identity/v3/projects/d01561573a044c87a4102522b8fe8432/users/5167ece674de4c11b71bb63db58f9eb3/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:52.507508 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.507790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.508558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.522374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-50d243ff-8039-445e-87bd-32e464510876 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:52.523739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-50d243ff-8039-445e-87bd-32e464510876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.528849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:52.530487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.579214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:52.613562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1071/2153] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:52.619600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d243ff-8039-445e-87bd-32e464510876 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.621107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-50d243ff-8039-445e-87bd-32e464510876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.641088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1083/2154] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:55:52.649163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.651770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.693944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d243ff-8039-445e-87bd-32e464510876 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.701294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.701961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.702947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.703765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.715175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.716819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.727173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833d21e-32e8-4f9a-aca2-9bf4a7b385d8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.728044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1072/2155] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:52.731805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.732425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.732804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.733078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.758722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.759019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310df083-a1ff-4ce6-af05-5f5da1ee9d2e service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.773980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1084/2156] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:52.779987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:52.781864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:52.806187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a502d9-16c0-4d3d-a184-27d559583f10 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:52.819414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:52.820140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.820695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.821231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] PATH_INFO: `/v3/projects/0bd41a96038443bc895d0b5d0ccf85ca` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.823014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd41a96038443bc895d0b5d0ccf85ca)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:52.825490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:52.888959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd41a96038443bc895d0b5d0ccf85ca', 'request_id': 'req-82e1c008-1acf-4f4e-b08a-0986886b5e3d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:52.897157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd41a96038443bc895d0b5d0ccf85ca', 'request_id': 'req-82e1c008-1acf-4f4e-b08a-0986886b5e3d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:52.904923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd41a96038443bc895d0b5d0ccf85ca', 'request_id': 'req-82e1c008-1acf-4f4e-b08a-0986886b5e3d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:52.918938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] The token cache is being invalidate because project 0bd41a96038443bc895d0b5d0ccf85ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:55:52.919790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:52.920532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-82e1c008-1acf-4f4e-b08a-0986886b5e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:55:52.922362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1073/2157] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:55:52 2022] DELETE /identity/v3/projects/0bd41a96038443bc895d0b5d0ccf85ca => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:55:52.927354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:52.927796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:52.928315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:52.952122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:52.953337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:53.020889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:53.033176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1085/2158] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:53.043933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.045337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.045784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.086795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:53.088026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:53.107962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1074/2159] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:53.114678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.127002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551bc3e-a23a-4011-aeb4-6cd8140deb95 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.132562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.133212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.133490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.133758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.134407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:55:53.134812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.139679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.156394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1086/2160] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:55:53.162001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.172957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786c452-76c6-492e-9717-3a8a144ce622 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.178371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.179030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.179295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.179966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.180182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.182761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.234430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1075/2161] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:53.239008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.259239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65d896-ffe4-4dbf-b9c5-9fd56a2efe57 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.361102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.361803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.362020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.362294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.362979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.367757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.421344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1087/2162] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:55:53.425549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.439171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef76f4d-7857-4413-ae38-7e672a9cbfd4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.444691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.445525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.446499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.446989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.447827 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.451951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.467596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1076/2163] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:53.472311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.486783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3f5a-dbd4-43df-8a44-b6022b6cedb3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.508714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.511344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.511869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.514257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.515402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.522161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb9d30-d499-446e-8590-675e6b213651 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.539293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1088/2164] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:53.543439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.556149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5ee7-ee67-4062-8a98-6e209d19a23c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.561381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.562305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.562753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.563081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] PATH_INFO: `/v3/projects/7d7cb1af46cd4442a44d678c03b276a9/users/2a706ab140ed40d9b295bcdd3c739bd7/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.588841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7cb1af46cd4442a44d678c03b276a9, user_id=2a706ab140ed40d9b295bcdd3c739bd7, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.592935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.612266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1077/2165] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:53 2022] PUT /identity/v3/projects/7d7cb1af46cd4442a44d678c03b276a9/users/2a706ab140ed40d9b295bcdd3c739bd7/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:53.627669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.628139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.628666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.673251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] MFA Rules not processed for user `2a706ab140ed40d9b295bcdd3c739bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:53.674776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:53.758100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1089/2166] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:53.764561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.776678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:53.836751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d047eb3-3991-4af9-a1dc-6991a17a77db admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.842139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.842768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.843048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.843325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.843958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.845844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.878590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1078/2167] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:53.880015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:53.891231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a568c0d-e7f7-48b9-8710-cb9462212fcc None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:53.943880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:53.956767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:53.957552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:53.957908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:53.958255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:53.958962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:53.960921 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:53.986127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1090/2168] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:53 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:55:53.990000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.002146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea3ad-4360-4a60-8081-4359680434f2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.007171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.007860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.008149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.008425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.009096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.010792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.027445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1079/2169] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:54.031366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.043894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b12074-225f-4104-8971-bfe36bc92515 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.050569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55055adb-7508-4467-a161-843072177e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.051178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055adb-7508-4467-a161-843072177e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.051456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055adb-7508-4467-a161-843072177e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.051725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055adb-7508-4467-a161-843072177e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.052375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55055adb-7508-4467-a161-843072177e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.055847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55055adb-7508-4467-a161-843072177e46 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.071604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1091/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:54.072717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.088385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757015f-7675-46ca-924a-b627b7cb7fe9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.094391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.094989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.095156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.095418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.095734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.096013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.096286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] PATH_INFO: `/v3/projects/947b511e4a1a45e6a15edacad20f928d/users/96e63d61a48245d58969f89b98d29b71/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.115822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947b511e4a1a45e6a15edacad20f928d, user_id=96e63d61a48245d58969f89b98d29b71, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.120584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dd698-0ba4-4e53-88f4-20af5e78ad36 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.152316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1080/2171] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:54 2022] PUT /identity/v3/projects/947b511e4a1a45e6a15edacad20f928d/users/96e63d61a48245d58969f89b98d29b71/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:54.162338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.166690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.169355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.178414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] MFA Rules not processed for user `5167ece674de4c11b71bb63db58f9eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:54.179712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:54.212447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] MFA Rules not processed for user `96e63d61a48245d58969f89b98d29b71`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:54.214661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:54.325255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1092/2172] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:54.334827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.336834 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:54.341424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1081/2173] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:54.346590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.420416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:54.431009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62019-ddb5-48c5-9e04-3eae453337a1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.436002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.443214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74f00-7df5-41ad-8458-f498b5fcae87 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.449784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.450199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.450484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.450623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.450790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.451025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.451365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.452096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.456139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.475099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.476676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.495767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1093/2174] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:55:54.507352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1082/2175] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:55:54 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:55:54.511360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.529164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a2c-6025-4ba8-ade0-b3c9fd0ca206 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.642420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.643206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.643591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.643963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.645423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.648537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.665774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:54.690254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1094/2176] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:55:54 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:55:54.692188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.705379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d80dc-ca2b-4ac0-9d3d-2ca7b60f09c2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.713123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.714828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.715694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.716157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.717101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.719049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.733571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1083/2177] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:54.737824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.750455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6618d1d-ebd5-4d5e-9f18-498bb07d1c92 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.758545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.759652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.760077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.760352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] PATH_INFO: `/v3/projects/fdff8d078fda46bea2c77c210a37f407/users/3348009581fa4bb8a1c363d7a47c3818/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.785306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdff8d078fda46bea2c77c210a37f407, user_id=3348009581fa4bb8a1c363d7a47c3818, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.791695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:54.823704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1095/2178] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:54 2022] PUT /identity/v3/projects/fdff8d078fda46bea2c77c210a37f407/users/3348009581fa4bb8a1c363d7a47c3818/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:54.828159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:54.841363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4217-ca41-4ea4-828b-13bce6ec8634 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:54.968543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:54.969220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:54.969454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:54.982322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:54.982322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:54.982322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28430e5-8d4c-4773-b8c6-beebd8312025 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:55.011250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1084/2179] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:55:55.016725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:55.035479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffc51a-680f-4f60-b6f7-8a9d5e696da1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:55.048268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:55.048994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:55.049377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:55.049882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] PATH_INFO: `/v3/projects/fdff8d078fda46bea2c77c210a37f407/users/3348009581fa4bb8a1c363d7a47c3818/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:55.058728 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdff8d078fda46bea2c77c210a37f407, user_id=3348009581fa4bb8a1c363d7a47c3818, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:55.063768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:55.099968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1096/2180] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:55:55 2022] PUT /identity/v3/projects/fdff8d078fda46bea2c77c210a37f407/users/3348009581fa4bb8a1c363d7a47c3818/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:55:55.110410 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e88be-b01d-49d7-b8ff-14049b41b175 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:55.110906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e88be-b01d-49d7-b8ff-14049b41b175 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:55.111180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e88be-b01d-49d7-b8ff-14049b41b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:55.146724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4a1e88be-b01d-49d7-b8ff-14049b41b175 None None] MFA Rules not processed for user `3348009581fa4bb8a1c363d7a47c3818`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:55.147968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4a1e88be-b01d-49d7-b8ff-14049b41b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:55.230569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1085/2181] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:55.244830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:55.314905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7843512-316a-4f94-abb0-bd4ffdd8cc78 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:55.320250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:55.320406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:55.320605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:55.320843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:55.410662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:55.413275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:55.426396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1097/2182] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:56.054372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:56.054826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:56.054946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:56.077260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] MFA Rules not processed for user `5167ece674de4c11b71bb63db58f9eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:55:56.077260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:55:56.134025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:55:56.162968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1086/2183] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:55:57.267678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:57.287952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3cab2-5d00-439b-bef3-21dc2cedcfef service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:57.293040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:57.293960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:57.304931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:57.307060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:57.483005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:57.484213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5c7b4-f3dc-4338-b7c9-259d8e50ec02 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:57.513041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1098/2184] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:55:58.513160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:55:58.536345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed88c4-945e-4620-8b87-168d19e5bf7e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:55:58.541262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:55:58.542108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:55:58.542564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:55:58.543539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:55:58.555753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:55:58.557687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:55:58.570375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1087/2185] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:55:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:56:01.066912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:01.067379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:01.067482 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:01.125194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:01.144586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] MFA Rules not processed for user `fb4bb9cd41cd46ddb87a746e732a948f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:01.146246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:01.411607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1099/2186] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:01.413421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:01.414981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:01.553314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e402-1212-4f31-ac8c-751035ea0e09 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:01.573239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:01.573239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:01.573239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:01.573239 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:01.574861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:01.574861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:01.622317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1088/2187] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:56:01 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:56:01.626023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:01.645596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ef764-d54a-4367-9cf8-9e78c425bbb2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:01.791345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:01.791990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:01.792456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:01.793207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:01.795196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:01.797847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:01.831130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1100/2188] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:56:01 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:56:01.842885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:01.864481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d81b0-2144-45dc-84af-1240ed5d331f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:01.871971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:01.872973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:01.873337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:01.874860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:01.875961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:01.877936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:01.897620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1089/2189] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:01.898947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:01.916478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ef3d4-3e72-4b6a-a440-03248075eac0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:01.927067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:01.928124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:01.928628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:01.929137 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] PATH_INFO: `/v3/projects/ef9b309814494508879ab2ab7b1d629c/users/8626dd696906488d8d950e6a82ff7235/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:01.954085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9b309814494508879ab2ab7b1d629c, user_id=8626dd696906488d8d950e6a82ff7235, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:01.957519 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:01.994592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1101/2190] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:56:01 2022] PUT /identity/v3/projects/ef9b309814494508879ab2ab7b1d629c/users/8626dd696906488d8d950e6a82ff7235/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:01.999377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:02.031102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175970-bb5d-4158-bffa-e97769afc97e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:02.162134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:02.162897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:02.163152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:02.163318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:02.163989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:02.165616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:02.177458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1090/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:02.181662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:02.196870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c218fc-335a-4394-90a3-72590b16fe31 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:02.204070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:02.204750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:02.205136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:02.205421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] PATH_INFO: `/v3/projects/ef9b309814494508879ab2ab7b1d629c/users/8626dd696906488d8d950e6a82ff7235/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:02.207531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9b309814494508879ab2ab7b1d629c, user_id=8626dd696906488d8d950e6a82ff7235, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:02.210263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b84b16-339d-41b2-8050-b0aefc64746b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:02.222420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1102/2192] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:56:02 2022] PUT /identity/v3/projects/ef9b309814494508879ab2ab7b1d629c/users/8626dd696906488d8d950e6a82ff7235/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:03.134766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:03.149559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168da7c9-0bd3-4ea6-8c7a-e0cfce272147 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:03.243083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:03.268348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:03.269074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:03.269356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:03.269682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] PATH_INFO: `/v3/users/2a706ab140ed40d9b295bcdd3c739bd7` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:03.271048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a706ab140ed40d9b295bcdd3c739bd7)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:03.274112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:03.362479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a706ab140ed40d9b295bcdd3c739bd7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:03.381640 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a706ab140ed40d9b295bcdd3c739bd7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:03.394628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a706ab140ed40d9b295bcdd3c739bd7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:03.416776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a706ab140ed40d9b295bcdd3c739bd7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:03.417533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a706ab140ed40d9b295bcdd3c739bd7'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:03.428694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1091/2193] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:56:03 2022] DELETE /identity/v3/users/2a706ab140ed40d9b295bcdd3c739bd7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:03.430437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:03.430981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:03.431200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:03.527951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:03.529713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:03.648531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:03.672903 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1103/2194] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:03.769561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:03.771065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:03.896648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62371b3-0f31-42ac-86d5-a4708ba56603 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:03.901060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:03.903302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:03.903302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:03.903675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:03.917092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:03.920076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:03.937146 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1092/2195] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:56:05.549373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:05.551050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:05.573066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217596c-798f-4ca0-995b-49abb7b763f6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:05.585187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:05.586850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:05.587334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:05.587802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] PATH_INFO: `/v3/projects/7d7cb1af46cd4442a44d678c03b276a9` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:05.589424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7cb1af46cd4442a44d678c03b276a9)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:05.598725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:05.697989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7cb1af46cd4442a44d678c03b276a9', 'request_id': 'req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:05.706933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7cb1af46cd4442a44d678c03b276a9', 'request_id': 'req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:05.718461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7cb1af46cd4442a44d678c03b276a9', 'request_id': 'req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:05.738174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] The token cache is being invalidate because project 7d7cb1af46cd4442a44d678c03b276a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:56:05.739152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:05.740047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:05.742261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1104/2196] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:56:05 2022] DELETE /identity/v3/projects/7d7cb1af46cd4442a44d678c03b276a9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:05.747913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:05.796858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975b941-57c8-446d-aa29-a755005287a9 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:05.944714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:05.945432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:05.945632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:05.945900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] PATH_INFO: `/v3/users/96e63d61a48245d58969f89b98d29b71` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:05.947330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e63d61a48245d58969f89b98d29b71)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:05.952297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:05.958986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:06.023107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e63d61a48245d58969f89b98d29b71'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:06.038653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e63d61a48245d58969f89b98d29b71'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:06.053749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e63d61a48245d58969f89b98d29b71'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:06.071911 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e63d61a48245d58969f89b98d29b71'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:06.075475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e63d61a48245d58969f89b98d29b71'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:06.093063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1093/2197] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:56:05 2022] DELETE /identity/v3/users/96e63d61a48245d58969f89b98d29b71 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:07.341717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:07.343977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:07.366374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606357f6-2fc0-49d8-9ca7-f9ff41fe0072 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:07.508765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:07.509452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:07.509892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:07.513207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] PATH_INFO: `/v3/projects/947b511e4a1a45e6a15edacad20f928d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:07.515407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947b511e4a1a45e6a15edacad20f928d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:07.517346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:07.589848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947b511e4a1a45e6a15edacad20f928d', 'request_id': 'req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.596981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947b511e4a1a45e6a15edacad20f928d', 'request_id': 'req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.604410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947b511e4a1a45e6a15edacad20f928d', 'request_id': 'req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.624568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] The token cache is being invalidate because project 947b511e4a1a45e6a15edacad20f928d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:56:07.625029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.625940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.628207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1105/2198] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:56:07 2022] DELETE /identity/v3/projects/947b511e4a1a45e6a15edacad20f928d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:07.632447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:07.635255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:07.660052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed28ec-1e62-40af-9be1-fbe46dc35572 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:07.757820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:07.758691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:07.759766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:07.760638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] PATH_INFO: `/v3/users/3348009581fa4bb8a1c363d7a47c3818` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:07.762841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3348009581fa4bb8a1c363d7a47c3818)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:07.766078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:07.883761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3348009581fa4bb8a1c363d7a47c3818'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.920007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3348009581fa4bb8a1c363d7a47c3818'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.932170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3348009581fa4bb8a1c363d7a47c3818'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.952163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3348009581fa4bb8a1c363d7a47c3818'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.953191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4aefb8e8-d770-4c23-a4fe-38d01dec29d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3348009581fa4bb8a1c363d7a47c3818'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:07.964025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1094/2199] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:56:07 2022] DELETE /identity/v3/users/3348009581fa4bb8a1c363d7a47c3818 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:08.339107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:08.341497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:08.362026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:08.380937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb6e1d-24f8-494c-a0a5-75bcc65a7963 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:08.502536 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:08.503531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:08.503993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:08.504274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] PATH_INFO: `/v3/projects/fdff8d078fda46bea2c77c210a37f407` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:08.505601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdff8d078fda46bea2c77c210a37f407)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:08.507680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:08.578352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdff8d078fda46bea2c77c210a37f407', 'request_id': 'req-5e9ce847-6bad-4dce-980f-251c4439a9c7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:08.588973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdff8d078fda46bea2c77c210a37f407', 'request_id': 'req-5e9ce847-6bad-4dce-980f-251c4439a9c7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:08.598312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdff8d078fda46bea2c77c210a37f407', 'request_id': 'req-5e9ce847-6bad-4dce-980f-251c4439a9c7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:08.615499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] The token cache is being invalidate because project fdff8d078fda46bea2c77c210a37f407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:56:08.615785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:08.616513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5e9ce847-6bad-4dce-980f-251c4439a9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:08.622662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1106/2200] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:56:08 2022] DELETE /identity/v3/projects/fdff8d078fda46bea2c77c210a37f407 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:08.625684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:08.625979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:08.626260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:08.639274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:08.659442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:08.661042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:08.739465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1095/2201] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:08.748823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:08.748823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:08.748823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:08.770944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:08.772193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:08.785455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1107/2202] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:08.786537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:08.800535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60604d-7814-4656-a9c1-17a48378f2c0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:08.806058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:08.806723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:08.807002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:08.807274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:08.807982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:56:08.808291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:08.812535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:08.846161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1096/2203] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:56:08.850959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:08.862552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf62a4-adc7-491a-8809-b95a168837fb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:08.875000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:08.875572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:08.875853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:08.876126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:08.876781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:08.878739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:08.960555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:08.963233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1108/2204] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:56:08 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:56:08.983797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0855b-ec97-4cfa-a32c-395775bf2b1e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:09.139833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:09.141061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.141499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.141973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.143466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:09.146294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:09.211416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1097/2205] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:56:08 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:56:09.215949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:09.227574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0755-eeff-4fe5-bfd0-91ddb291071f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:09.233760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:09.236101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.236399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.236641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.237292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:09.241707 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:09.263929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1109/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:09.268488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:09.280506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abde172-1273-4cbf-96d8-0d8e22fec256 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:09.291269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:09.291914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.292192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.292501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.293513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:09.295495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a4dbb-3bd4-4f40-baf5-fa86fdf98c0b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:09.307885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1098/2207] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:09.312249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:09.324926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb52a30-dc85-4922-bffc-3474845e5c3f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:09.331397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:09.332314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.333355 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.333817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] PATH_INFO: `/v3/projects/ad1e93f4ab76409681e8b78a0f5050d5/users/38ae036ac8d9458a978d5d7f640723de/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.359214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1e93f4ab76409681e8b78a0f5050d5, user_id=38ae036ac8d9458a978d5d7f640723de, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:09.361961 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:09.380152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1110/2208] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:56:09 2022] PUT /identity/v3/projects/ad1e93f4ab76409681e8b78a0f5050d5/users/38ae036ac8d9458a978d5d7f640723de/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:56:09.391547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abcf-e4b5-4505-8059-aa7e71920cfc None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.391832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abcf-e4b5-4505-8059-aa7e71920cfc None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.392103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abcf-e4b5-4505-8059-aa7e71920cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.432542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-64f4abcf-e4b5-4505-8059-aa7e71920cfc None None] MFA Rules not processed for user `38ae036ac8d9458a978d5d7f640723de`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:09.434392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-64f4abcf-e4b5-4505-8059-aa7e71920cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:09.506061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:09.512128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1099/2209] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:09.523651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:09.525340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:09.561993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:09.588862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7f61f-ee29-40d5-8b6a-600d5bc50f52 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:09.592722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:09.593335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:09.593616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:09.594133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:09.606346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:09.607910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:09.617321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1111/2210] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:56:10.059443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:10.060032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:10.060032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:10.089405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] MFA Rules not processed for user `8626dd696906488d8d950e6a82ff7235`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:10.090657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:10.160027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1100/2211] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:10.174206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:10.175641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:10.249025 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd69e5-3709-4b54-857f-ff40bc9212d0 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:10.254690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:10.255294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:10.255572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:10.255864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:10.257559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:10.319824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:10.321548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e011a3-f28b-4193-95ff-ee35993200fe service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:10.336579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1112/2212] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:56:12.344052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:12.431530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350d28-4b62-4a72-8253-e79cdb3ad7ba None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:12.437100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:12.437750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:12.438033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:12.438309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] PATH_INFO: `/v3/users/38ae036ac8d9458a978d5d7f640723de` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:12.439519 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ae036ac8d9458a978d5d7f640723de)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:12.441739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:12.520518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ae036ac8d9458a978d5d7f640723de'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:12.541044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ae036ac8d9458a978d5d7f640723de'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:12.554581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ae036ac8d9458a978d5d7f640723de'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:12.567215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ae036ac8d9458a978d5d7f640723de'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:12.568226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ae036ac8d9458a978d5d7f640723de'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:12.579627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1101/2213] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:56:12 2022] DELETE /identity/v3/users/38ae036ac8d9458a978d5d7f640723de => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:12.584240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:12.584755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:12.584876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:12.612202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:12.613607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:12.692945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1113/2214] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:56:12.703452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:12.704870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:12.713743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:12.791570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be6a1-334c-4d24-8a82-3023f1762f9a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:12.795825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:12.796435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:12.796714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:12.796982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:12.806364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:12.808895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cd7fc-a3a4-41ee-bd12-f99520b5b100 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:12.833926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1102/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:56:13.453056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:13.454592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:13.470481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e818096-51ce-442c-bd36-d13203c40542 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:13.479114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:13.480342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:13.480625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:13.480902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] PATH_INFO: `/v3/projects/ad1e93f4ab76409681e8b78a0f5050d5` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:13.482381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1e93f4ab76409681e8b78a0f5050d5)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:13.484424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:13.492116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:13.541004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1e93f4ab76409681e8b78a0f5050d5', 'request_id': 'req-24fc5ec6-06dc-43af-81fb-fe4439509088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:13.548167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1e93f4ab76409681e8b78a0f5050d5', 'request_id': 'req-24fc5ec6-06dc-43af-81fb-fe4439509088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:13.554973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1e93f4ab76409681e8b78a0f5050d5', 'request_id': 'req-24fc5ec6-06dc-43af-81fb-fe4439509088'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:13.569578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] The token cache is being invalidate because project ad1e93f4ab76409681e8b78a0f5050d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:56:13.569774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:13.570948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-24fc5ec6-06dc-43af-81fb-fe4439509088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:56:13.577533 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1114/2216] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:56:13 2022] DELETE /identity/v3/projects/ad1e93f4ab76409681e8b78a0f5050d5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:13.579153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:13.579597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:13.579680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:13.602749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:13.604147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:13.673141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1103/2217] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:13.681223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:13.683713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:13.683713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:13.709130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:13.710634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:13.722714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1115/2218] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:13.726251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:13.746914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cd3b4-0cf8-4b65-ad4c-08a70cf8b490 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:13.754898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:13.755744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:13.756205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:13.756949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:13.758018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:56:13.758492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:13.760194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:13.781573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1104/2219] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 11:56:13.787858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:13.803264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713f29-860e-448d-90e3-0e98624dc22d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:13.809856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:13.810856 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:13.811363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:13.811803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:13.812457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:13.814410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:13.851427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1116/2220] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:56:13 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:56:13.857305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:13.867411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fca975-05ac-4814-950d-c415a05791be admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:14.012743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:14.017494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.017851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.018201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.018997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:14.022753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:14.077623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1105/2221] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:56:13 2022] POST /identity/v3/users => generated 282 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:56:14.078734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:14.106417 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19aa64-3a5d-409c-8d12-5a55cdf39689 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:14.114051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:14.114725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.115007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.115329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.115993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:14.118009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:14.133731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1117/2222] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:14.138377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:14.157993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed533-11a4-4baf-95b2-9fb2f46bb608 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:14.168100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:14.168759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.174427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.174809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.175530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:14.177181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fbc38-e9f3-4be4-bf40-399944c49933 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:14.189668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1106/2223] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:56:14.194123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:14.205309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a333257-a8aa-4271-9e56-2c074aead4d7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:14.210647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:14.211443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.211807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.212160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] PATH_INFO: `/v3/projects/7ee9a343284445f2ad173a7318bd978b/users/38a1ba44318447f496c69506c1b7ab2a/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.229196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee9a343284445f2ad173a7318bd978b, user_id=38a1ba44318447f496c69506c1b7ab2a, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:14.232343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:14.247456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1118/2224] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:56:14 2022] PUT /identity/v3/projects/7ee9a343284445f2ad173a7318bd978b/users/38a1ba44318447f496c69506c1b7ab2a/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:56:14.257765 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.258081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.258361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.288896 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:14.293634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] MFA Rules not processed for user `38a1ba44318447f496c69506c1b7ab2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:14.295925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:14.389824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1107/2225] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:14.402568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:14.405364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:14.478959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0b197-f2d1-41f9-a382-f0e2a78be518 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:14.485465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:14.486105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:14.486391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:14.486708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:14.499539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:14.499539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129d8e5-6e83-46c3-b643-752c74d9a029 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:14.510615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1119/2226] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:56:15.865969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:56:15.868274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:15.951384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0fea3-8c0d-4cf2-8ff6-feb31ccad7bd None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:56:15.956675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:56:15.957300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:15.957599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:15.957864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:15.959692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:15.971438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:56:15.973035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d8d5a-b343-4349-9606-f0e0b5b5dcab service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:56:15.984497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1108/2227] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:56:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:56:24.507293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:25.979167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=58 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:35.172862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf9b4-fc3e-48fc-9c8b-d9d865276964 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:35.173392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf9b4-fc3e-48fc-9c8b-d9d865276964 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:35.173570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf9b4-fc3e-48fc-9c8b-d9d865276964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:35.194200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5efbf9b4-fc3e-48fc-9c8b-d9d865276964 None None] MFA Rules not processed for user `5167ece674de4c11b71bb63db58f9eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:35.195520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5efbf9b4-fc3e-48fc-9c8b-d9d865276964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:35.256826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1120/2228] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:36.678975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8542c-83e8-4b25-ba23-49e9a766aa80 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:56:36.680401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8542c-83e8-4b25-ba23-49e9a766aa80 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:56:36.681165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8542c-83e8-4b25-ba23-49e9a766aa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:56:36.704662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-9ec8542c-83e8-4b25-ba23-49e9a766aa80 None None] MFA Rules not processed for user `66682d5656d3414eaebba666d4e0771f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:56:36.706015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9ec8542c-83e8-4b25-ba23-49e9a766aa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:56:36.783069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1109/2229] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:56:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:56:45.249696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=25 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:56:46.774240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=25 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:01.363821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:01.364273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:01.364375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:01.390373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] MFA Rules not processed for user `fb4bb9cd41cd46ddb87a746e732a948f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:01.391684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:01.462047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1121/2230] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:11.454277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=25 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:19.373952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:19.374441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:19.374561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:19.394193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:19.395448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:19.452383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1110/2231] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:19.493814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:19.495244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:19.568595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c102b-adca-4301-bcb0-08a0b7521fb2 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:19.573617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:19.575114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:19.575257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:19.575505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:19.583870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:19.585443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d02a8-6b11-4c06-b500-69280646ea85 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:19.594779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1122/2232] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:57:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:57:29.445223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=25 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:29.591256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=32 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:36.705704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:36.809502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe0711-2ea6-4c70-94e2-29b59f2ffdf1 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:36.817608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:36.817943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:36.818343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:36.818735 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] PATH_INFO: `/v3/users/38a1ba44318447f496c69506c1b7ab2a` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:36.820266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a1ba44318447f496c69506c1b7ab2a)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:36.824277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:36.881962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a1ba44318447f496c69506c1b7ab2a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:36.898538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a1ba44318447f496c69506c1b7ab2a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:36.908632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a1ba44318447f496c69506c1b7ab2a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:36.921069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a1ba44318447f496c69506c1b7ab2a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:36.922124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a1ba44318447f496c69506c1b7ab2a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:36.930594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1111/2233] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:57:36 2022] DELETE /identity/v3/users/38a1ba44318447f496c69506c1b7ab2a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:36.935219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d376b-be66-4148-a886-1b17478eddde None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:36.935718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d376b-be66-4148-a886-1b17478eddde None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:36.935801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d376b-be66-4148-a886-1b17478eddde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:36.970190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-469d376b-be66-4148-a886-1b17478eddde None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:36.971512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-469d376b-be66-4148-a886-1b17478eddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:37.061529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1123/2234] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:37.069230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:37.070707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:37.166097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b9769-3ffa-4a34-bc6f-b533ffbcd496 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:37.178210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:37.178998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:37.179486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:37.180453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:37.193447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:37.195025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508868be-7f7e-4a46-87af-3e1154649fa6 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:37.209603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1112/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:57:37.704431 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d376b-be66-4148-a886-1b17478eddde None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:37.712875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-469d376b-be66-4148-a886-1b17478eddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:37.734176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d376b-be66-4148-a886-1b17478eddde None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:37.751753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:37.753102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:37.754160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:37.759196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] PATH_INFO: `/v3/projects/7ee9a343284445f2ad173a7318bd978b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:37.761650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee9a343284445f2ad173a7318bd978b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:37.763815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:37.836432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee9a343284445f2ad173a7318bd978b', 'request_id': 'req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:37.844314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee9a343284445f2ad173a7318bd978b', 'request_id': 'req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:37.854779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee9a343284445f2ad173a7318bd978b', 'request_id': 'req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:37.897925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] The token cache is being invalidate because project 7ee9a343284445f2ad173a7318bd978b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:57:37.898257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:37.899060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:37.915253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1124/2236] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:57:37 2022] DELETE /identity/v3/projects/7ee9a343284445f2ad173a7318bd978b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:42.926932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:42.927449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:42.927548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:42.947941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:42.949430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:42.967227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:43.006776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1113/2237] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 11:57:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:43.019400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:43.020797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:43.090726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0819b47-e0ff-46a2-b0e5-b1eebca7fca6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:43.095690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:43.096292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:43.096570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:43.096842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:43.105279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:43.106899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:43.119964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1125/2238] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:57:45.279554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:45.281657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:45.380819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb921d3-7017-498b-a81b-a429a1e4972c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:45.386372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:45.387803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:45.388153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:45.388570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] PATH_INFO: `/v3/users/5167ece674de4c11b71bb63db58f9eb3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:45.389807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5167ece674de4c11b71bb63db58f9eb3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:45.392250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:45.449771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5167ece674de4c11b71bb63db58f9eb3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.467749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5167ece674de4c11b71bb63db58f9eb3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.476405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5167ece674de4c11b71bb63db58f9eb3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.485965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5167ece674de4c11b71bb63db58f9eb3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.487372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e583ecb-636f-43db-9b2f-2ed2a200b45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5167ece674de4c11b71bb63db58f9eb3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.496950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1114/2239] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:57:45 2022] DELETE /identity/v3/users/5167ece674de4c11b71bb63db58f9eb3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:45.685395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:45.702928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e76-bb3d-4432-9e79-5958f1dbeea9 service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:45.716673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:45.773585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:45.774216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:45.774521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:45.774799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] PATH_INFO: `/v3/projects/d01561573a044c87a4102522b8fe8432` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:45.776087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01561573a044c87a4102522b8fe8432)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:45.777980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:45.856754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01561573a044c87a4102522b8fe8432', 'request_id': 'req-5114feac-413a-46e1-a217-288dbfbac35c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.871787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01561573a044c87a4102522b8fe8432', 'request_id': 'req-5114feac-413a-46e1-a217-288dbfbac35c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.881625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01561573a044c87a4102522b8fe8432', 'request_id': 'req-5114feac-413a-46e1-a217-288dbfbac35c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.906027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] The token cache is being invalidate because project d01561573a044c87a4102522b8fe8432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:57:45.906027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.906736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5114feac-413a-46e1-a217-288dbfbac35c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:45.911327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1126/2240] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:57:45 2022] DELETE /identity/v3/projects/d01561573a044c87a4102522b8fe8432 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:45.913052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:45.913381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:45.913613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:45.935753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:45.938324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:45.995985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:46.057643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1115/2241] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:46.066438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.066882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.067241 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.103747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:46.107898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:46.126206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1127/2242] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:46.130274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.140142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe8932-8405-4d4b-8324-bbe8a698d2db None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.144995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.145541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.145820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.146091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.146790 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:57:46.147096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.148709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.162721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1116/2243] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:57:46.167017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.178774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dff82-81a5-4828-a2b7-7d1c482bd02c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.185534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.186398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.186994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.187441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.188273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.191310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2e292-6d75-47b1-831b-c9cf47e49a68 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.231607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1128/2244] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:57:46.235985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.248328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d9d2-ed18-4446-bc51-90269ddcce20 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.324476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.325931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.326381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.326839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.328026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.330708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.362499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1117/2245] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:57:46.362841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a8d9-10c5-4907-8dd6-7be66969c988 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.363660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a8d9-10c5-4907-8dd6-7be66969c988 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.363660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a8d9-10c5-4907-8dd6-7be66969c988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.366303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.378411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d331-bd22-4f10-8706-a1f5d9bb2757 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.384290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.384973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.385722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.385722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.386676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.387160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3309a8d9-10c5-4907-8dd6-7be66969c988 None None] MFA Rules not processed for user `f58a59c389cf4b3198a6c452eb2772d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:46.388290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.388635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3309a8d9-10c5-4907-8dd6-7be66969c988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:46.403483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1118/2246] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 11:57:46.405333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.415799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfbf1-2d8d-48c7-a69f-2fc27257234b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.420825 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.421505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.421858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.422356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.423105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.424773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.440103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1119/2247] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:57:46.441212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.451727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2067ed-aaf2-4534-9246-3e929becb2ed admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.457024 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.457370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1129/2248] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:57:46.457748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.458034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.458308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] PATH_INFO: `/v3/projects/4a5205d95c8a4f38baeee576c198f6d1/users/faf0b50485284750af111da511622733/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.474614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5205d95c8a4f38baeee576c198f6d1, user_id=faf0b50485284750af111da511622733, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.477356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.491421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1120/2249] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:57:46 2022] PUT /identity/v3/projects/4a5205d95c8a4f38baeee576c198f6d1/users/faf0b50485284750af111da511622733/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:57:46.502046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.502334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.502642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.530846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] MFA Rules not processed for user `faf0b50485284750af111da511622733`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:46.532053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:46.540487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:46.602670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1130/2250] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:46.615123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.616913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:46.686562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08522174-dee0-4808-82ea-aaf2a1a76435 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.690832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.691944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.691944 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.692168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.702062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.703544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb1012-75a0-41fa-8ce8-2371a8975044 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.713247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1121/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:57:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:57:46.740276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:46.810337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63a09-9771-4cab-af20-1e1f343efbe1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:46.815564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:46.816269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.816630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.816987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] PATH_INFO: `/v3/users/faf0b50485284750af111da511622733` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.818180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf0b50485284750af111da511622733)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:46.820205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:46.869806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf0b50485284750af111da511622733'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:46.876607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf0b50485284750af111da511622733'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:46.889406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf0b50485284750af111da511622733'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:46.890290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf0b50485284750af111da511622733'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:46.898848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf0b50485284750af111da511622733'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:46.904861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1131/2252] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:57:46 2022] DELETE /identity/v3/users/faf0b50485284750af111da511622733 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:46.909312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:46.909685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:46.910031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:46.929403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:46.930739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:46.944605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:46.997100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1122/2253] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:57:47.006240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.007924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:47.043334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:47.073191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1eee7c-c7f3-458a-b762-c3c8d0ba2535 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:47.077497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:47.078136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.078412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.078696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.095908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:47.096699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1ec1b-380e-47da-8311-7e02f8029ea6 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:47.113054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1132/2254] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 11:57:47.529287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.542806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:47.550460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599de3c0-4eed-4bab-9215-410f0fed1ce3 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:47.555584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:47.556232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.557333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.557774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] PATH_INFO: `/v3/projects/4a5205d95c8a4f38baeee576c198f6d1` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.559189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5205d95c8a4f38baeee576c198f6d1)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:47.561365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:47.613705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5205d95c8a4f38baeee576c198f6d1', 'request_id': 'req-11aa8c13-5140-42b2-a978-5c021a73b070'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:47.620798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5205d95c8a4f38baeee576c198f6d1', 'request_id': 'req-11aa8c13-5140-42b2-a978-5c021a73b070'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:47.626354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5205d95c8a4f38baeee576c198f6d1', 'request_id': 'req-11aa8c13-5140-42b2-a978-5c021a73b070'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:47.637282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] The token cache is being invalidate because project 4a5205d95c8a4f38baeee576c198f6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:57:47.637505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:47.638127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-11aa8c13-5140-42b2-a978-5c021a73b070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:57:47.639625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1123/2255] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:57:47 2022] DELETE /identity/v3/projects/4a5205d95c8a4f38baeee576c198f6d1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:47.644509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.644794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.645071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.664440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:47.665625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:47.724122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1133/2256] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:47.732422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.732706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.733004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.752392 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:47.753579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:47.761562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1124/2257] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:47.765692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.775714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bb0ef-8f00-45a8-9407-e33ecba9be58 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:47.783562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:47.784246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.784531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.784840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.785602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 11:57:47.785794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:47.787495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:47.803901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1134/2258] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 11:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 11:57:47.805468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.815901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c8755-89fd-4bf6-bffb-4ce489a86a51 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:47.820603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:47.821267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.821548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.821841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.823264 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:47.825210 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:47.855863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1125/2259] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 11:57:47 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 11:57:47.860388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.872194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c276-6145-4a27-b4c0-515f9110c72b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:47.951420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:47.952043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:47.952316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:47.952585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:47.953217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:47.955200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:47.986272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1135/2260] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 11:57:47 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 11:57:47.987858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:47.999963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c301-58a2-4cf4-a54f-09f992fee03a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:48.004565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:48.005165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:48.005443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:48.005711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:48.006543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:48.007992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:48.018056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1126/2261] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:57:48.022818 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:48.033916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0cbd-d251-42db-9203-097c7ebc2130 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:48.038717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:48.039320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:48.039602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:48.039879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:48.040534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:48.042951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a57f-bc19-41dc-904e-6f5c2ddf1d40 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:48.053736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1136/2262] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 11:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 11:57:48.058574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:48.068948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5cd72-51a4-45a3-a2ab-79df6c1c340f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:48.076890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:48.077566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:48.077850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:48.078127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] PATH_INFO: `/v3/projects/e6edca5cca3e43e8835b6382e2d5ec1c/users/f2faaf7fcb0142b6b7aeb68f7e675e33/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:48.098830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6edca5cca3e43e8835b6382e2d5ec1c, user_id=f2faaf7fcb0142b6b7aeb68f7e675e33, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:48.101549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:48.123253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1127/2263] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 11:57:48 2022] PUT /identity/v3/projects/e6edca5cca3e43e8835b6382e2d5ec1c/users/f2faaf7fcb0142b6b7aeb68f7e675e33/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:57:48.196406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11212-9e38-4154-b7ab-39d8287eca81 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:48.196694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11212-9e38-4154-b7ab-39d8287eca81 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:48.197673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11212-9e38-4154-b7ab-39d8287eca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:48.230737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d0c11212-9e38-4154-b7ab-39d8287eca81 None None] MFA Rules not processed for user `f2faaf7fcb0142b6b7aeb68f7e675e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:48.232076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d0c11212-9e38-4154-b7ab-39d8287eca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:48.267981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:48.337325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1137/2264] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:57:48.351798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:48.353916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:48.439325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:57:48.463583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec36-a0c6-4745-b74d-1a3d2f9d94e8 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:48.467708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:48.468329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:48.468610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:48.468883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:48.478726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:48.480759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:48.489858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1128/2265] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:57:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:57:50.964064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e9df-2032-4d24-8a12-0b8b76ba206a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:50.964769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e9df-2032-4d24-8a12-0b8b76ba206a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:50.965410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e9df-2032-4d24-8a12-0b8b76ba206a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:50.993967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f2b9e9df-2032-4d24-8a12-0b8b76ba206a None None] MFA Rules not processed for user `f2faaf7fcb0142b6b7aeb68f7e675e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:50.996018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f2b9e9df-2032-4d24-8a12-0b8b76ba206a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:51.004689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1138/2266] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:57:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:57:52.097508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:52.101413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:52.214645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f0122-2804-4c29-8962-9361de78438a service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:52.219792 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:52.220592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:52.221053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:52.221698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:52.224167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:52.308049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:52.310651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:52.324357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1129/2267] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:57:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:57:52.661527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:52.662102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:52.662102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:52.696958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:57:52.698237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:52.831030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1139/2268] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:57:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:57:52.838435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:57:52.840153 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:57:52.993510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4754-a0a8-45f4-bf85-b31915a6fc9a service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:57:53.001395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:57:53.002140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:57:53.002532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:57:53.002906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:57:53.012669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:57:53.014713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:57:53.028990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1130/2269] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:58:00.457353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:00.459063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:00.472591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f257-6548-4411-a230-bbfae17be8d0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:00.477786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:00.478888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:00.479170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:00.479501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:00.481235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:00.574531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:00.576275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932bdd0-3779-4b79-9de0-a5ada3f39f53 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:00.588640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1140/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:02.540695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:02.551608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b43c1e-a1ae-4304-bcf3-8071f75e9e3c service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:02.558607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:02.559242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:02.559526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:02.559797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:02.561838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:02.576330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:02.578058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:02.590415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1131/2271] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:05.639671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accbdc-a66e-400f-839c-da479010fac0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:05.640225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accbdc-a66e-400f-839c-da479010fac0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:05.640225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accbdc-a66e-400f-839c-da479010fac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:05.659890 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a6accbdc-a66e-400f-839c-da479010fac0 None None] MFA Rules not processed for user `f4b71a4e5c1a4656a541f9cacec0af18`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:05.661098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a6accbdc-a66e-400f-839c-da479010fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:05.709661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:05.743392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1141/2272] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:58:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:10.003330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:10.004817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:10.018493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68e753-d3a5-4d9a-b564-aade33970c1d service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:10.023979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:10.024577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:10.024857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:10.025125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:10.026858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:10.033684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:10.090258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:10.092481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1bee-c1b3-4b8a-aee5-d00d18660d93 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:10.102334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1132/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:15.718867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:17.213950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.214608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.214708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.216075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1142/2274] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:58:17.223135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255420f-bddd-4255-8f03-41190a0a8e41 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.223704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255420f-bddd-4255-8f03-41190a0a8e41 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.223910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255420f-bddd-4255-8f03-41190a0a8e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.245693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-7255420f-bddd-4255-8f03-41190a0a8e41 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:17.246958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7255420f-bddd-4255-8f03-41190a0a8e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:17.257140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1133/2275] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:17.269346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:17.271090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:17.330327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed82b0-e260-43e9-ade9-5798a867f9c9 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:17.333906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:17.334541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.334823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.335093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.345462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:17.346924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c579762-7902-42c8-929a-ae25b9c78e84 service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:17.357996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1143/2276] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:17.426738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f778686-3051-4e3b-ac07-3de8beb8fcd4 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.427030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f778686-3051-4e3b-ac07-3de8beb8fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.427326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f778686-3051-4e3b-ac07-3de8beb8fcd4 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.428686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1134/2277] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:58:17.433569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28551d-958b-43df-a7f5-1de45f84b9c1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.433884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28551d-958b-43df-a7f5-1de45f84b9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.434165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28551d-958b-43df-a7f5-1de45f84b9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.454441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ad28551d-958b-43df-a7f5-1de45f84b9c1 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:17.455674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ad28551d-958b-43df-a7f5-1de45f84b9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:17.462202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1144/2278] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:17.474317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f778686-3051-4e3b-ac07-3de8beb8fcd4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:17.485871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f778686-3051-4e3b-ac07-3de8beb8fcd4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:17.489372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:17.489967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.490245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.490540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.497608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:17.499078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2d79e-b87d-4574-89a3-aba100fc4167 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:17.507483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1135/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:17.593030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340f49-38f8-4716-b7ff-89e7668b4e19 None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.593322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340f49-38f8-4716-b7ff-89e7668b4e19 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.593602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340f49-38f8-4716-b7ff-89e7668b4e19 None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.595029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1145/2280] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 11:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 11:58:17.600041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.600329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.600603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.620467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:17.621649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:17.630927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1136/2281] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:17.641221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb340f49-38f8-4716-b7ff-89e7668b4e19 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:17.652272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb340f49-38f8-4716-b7ff-89e7668b4e19 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:17.656472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:17.657064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:17.657347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:17.657624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:17.667777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:17.669198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf7006-7f36-46cf-9528-c08274081480 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:17.682983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1146/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:27.615583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=36 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:27.681047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=35 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:28.337573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:28.347482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69544f0a-fd54-428b-8b39-d7f5a7b359f2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:28.351111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:28.351723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:28.352003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:28.352301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:28.354034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:28.364402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:28.365889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:28.375486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1137/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:28.858756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd432b-f06d-476f-bc8b-81c1e7409841 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:28.860048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd432b-f06d-476f-bc8b-81c1e7409841 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:28.860048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd432b-f06d-476f-bc8b-81c1e7409841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:28.881416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-0dbd432b-f06d-476f-bc8b-81c1e7409841 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:28.882727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0dbd432b-f06d-476f-bc8b-81c1e7409841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:28.890133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1147/2284] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:58:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:28.913638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:28.926404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c27d0-a68a-40ce-9626-f1660d5e0fae service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:28.932315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:28.932922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:28.933197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:28.933467 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:28.943127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:28.944927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:28.953198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1138/2285] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:58:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:58:38.876204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:38.951352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=12 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:52.180575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24d91-b2fb-45ee-9240-bb8e7766e3f8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:52.181107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24d91-b2fb-45ee-9240-bb8e7766e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:52.181528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d24d91-b2fb-45ee-9240-bb8e7766e3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:52.203824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-71d24d91-b2fb-45ee-9240-bb8e7766e3f8 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:52.205441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-71d24d91-b2fb-45ee-9240-bb8e7766e3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:52.213612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1148/2286] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:52.226980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:52.237752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc39-b708-4398-9343-4d8605acb023 service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:52.243368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:52.243976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:52.244261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:52.244535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:52.253847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:52.255577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:52.265282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1139/2287] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:58:52.788303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a987-edc2-4e5b-a54b-8979a12b54f1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:52.788591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a987-edc2-4e5b-a54b-8979a12b54f1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:52.788912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3a987-edc2-4e5b-a54b-8979a12b54f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:52.809521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-98c3a987-edc2-4e5b-a54b-8979a12b54f1 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:52.810773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-98c3a987-edc2-4e5b-a54b-8979a12b54f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:52.818254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1149/2288] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:58:52.830921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:52.841795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd819db-1fc7-460d-a12c-6de69d13b94b service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:52.847532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:52.848132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:52.848417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:52.848688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:52.857055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:52.859607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:52.868123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1140/2289] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:58:53.128971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99ac83-1d4a-4c82-8d6d-c4e6e0b9af8f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:53.129274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99ac83-1d4a-4c82-8d6d-c4e6e0b9af8f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:53.129684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99ac83-1d4a-4c82-8d6d-c4e6e0b9af8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:53.161394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ac99ac83-1d4a-4c82-8d6d-c4e6e0b9af8f None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:53.162722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ac99ac83-1d4a-4c82-8d6d-c4e6e0b9af8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:53.170315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1150/2290] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:53.184380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:53.197050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2eef-6bf3-4ea4-a27b-758793cd4eed service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:53.204729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:53.205560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:53.206020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:53.206466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:53.215503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:53.217142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:53.225724 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1141/2291] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 11:58:53.549665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43608ad-6409-4462-8940-5e63903b214e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:53.550131 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43608ad-6409-4462-8940-5e63903b214e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:53.550226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43608ad-6409-4462-8940-5e63903b214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:53.569164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f43608ad-6409-4462-8940-5e63903b214e None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:53.570270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f43608ad-6409-4462-8940-5e63903b214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:53.577350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1151/2292] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:58:53.589726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:53.601069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb052e-a570-4197-ac8c-5e9997357633 service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:53.606011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:53.606634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:53.606920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:53.607213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:53.616108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:53.617661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbcc28-7a4b-414c-95f8-9a8ad2f9b2a7 service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:53.625537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1142/2293] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:58:55.735263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43608ad-6409-4462-8940-5e63903b214e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:55.799043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43608ad-6409-4462-8940-5e63903b214e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:55.803460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:55.804081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:55.804356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:55.804624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] PATH_INFO: `/v3/users/f2faaf7fcb0142b6b7aeb68f7e675e33` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:55.805731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2faaf7fcb0142b6b7aeb68f7e675e33)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:55.807663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:55.853716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2faaf7fcb0142b6b7aeb68f7e675e33'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:55.861204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2faaf7fcb0142b6b7aeb68f7e675e33'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:55.873319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2faaf7fcb0142b6b7aeb68f7e675e33'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:55.873971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2faaf7fcb0142b6b7aeb68f7e675e33'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:55.880769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2faaf7fcb0142b6b7aeb68f7e675e33'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:55.886456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1152/2294] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 11:58:55 2022] DELETE /identity/v3/users/f2faaf7fcb0142b6b7aeb68f7e675e33 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 11:58:55.891006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:55.891506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:55.891606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:55.910746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:58:55.911924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:55.970491 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1143/2295] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 11:58:55.980875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:55.982305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:56.041348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373b96a-9fab-4b4e-9ac4-a5ca6f90cb64 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:56.045007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:56.045710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:56.045991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:56.046264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:56.053199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:58:56.055544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:56.057419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:56.068510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1153/2296] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:58:56.233520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:58:56.234979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:58:56.247189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11017-cbd9-49d9-8178-2f5243a3e497 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:58:56.252156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:58:56.252802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:58:56.253081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:58:56.253385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] PATH_INFO: `/v3/projects/e6edca5cca3e43e8835b6382e2d5ec1c` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:58:56.255082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6edca5cca3e43e8835b6382e2d5ec1c)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:58:56.256930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:58:56.320319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6edca5cca3e43e8835b6382e2d5ec1c', 'request_id': 'req-ed2577de-b29f-4e72-9301-75842cb8ef1c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:56.329871 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6edca5cca3e43e8835b6382e2d5ec1c', 'request_id': 'req-ed2577de-b29f-4e72-9301-75842cb8ef1c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:56.336497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6edca5cca3e43e8835b6382e2d5ec1c', 'request_id': 'req-ed2577de-b29f-4e72-9301-75842cb8ef1c'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:56.353168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] The token cache is being invalidate because project e6edca5cca3e43e8835b6382e2d5ec1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 11:58:56.353456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:56.354113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ed2577de-b29f-4e72-9301-75842cb8ef1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 11:58:56.355637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1144/2297] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 11:58:56 2022] DELETE /identity/v3/projects/e6edca5cca3e43e8835b6382e2d5ec1c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 11:59:02.739257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:59:02.741053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:02.831352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6771-9dcd-42f9-be23-7c10713ebf19 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:59:02.893463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:59:02.894074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:02.894350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:02.894654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:02.896358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:02.908742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:59:02.910301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:59:02.921287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1154/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:59:06.187757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:06.188227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:06.188348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:06.208166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:59:06.209386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:06.236471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:59:06.267143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1145/2299] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 11:59:06.280506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:59:06.281885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:06.351127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd70a9-5031-4fa0-8eb9-68b7ad9588ce service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:59:06.356102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:59:06.356704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:06.356986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:06.357939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:06.365943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:59:06.367729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:59:06.375931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1155/2300] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 11:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 11:59:16.260149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=11 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:59:16.373258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:59:20.146066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:59:20.147495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:20.165423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6696be-d806-4125-9b67-ff909efe3ea5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:59:20.173678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:59:20.174302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:20.174622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:20.174889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:20.176628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:20.234494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:59:20.236107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ac944-ffb1-449e-967a-9522c6f70ac1 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:59:20.245086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1146/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:59:30.240878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=28 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:59:33.843102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 11:59:33.845048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:33.862082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43554b5f-b3bb-4a5a-9aed-3942fb026bde service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 11:59:33.868027 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 11:59:33.868700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:33.868879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:33.869103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:33.870940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:33.883511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 11:59:33.885072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 11:59:33.895855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1156/2302] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 11:59:43.891129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 11:59:56.860994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 11:59:56.861521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 11:59:56.861521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 11:59:56.880800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] MFA Rules not processed for user `66682d5656d3414eaebba666d4e0771f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 11:59:56.881992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 11:59:56.888847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1147/2303] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:00:03.237196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:03.261610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1082f75-2c6a-4c28-a4c2-40b6d7b4c6e0 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:03.268908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:03.269703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:03.270177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:03.270643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:03.272553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:03.291344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:03.293783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:03.305796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1157/2304] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:03.734713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:03.736727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:03.812379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4e55-0535-4f38-a45f-c54277c2aa4b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:03.817342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:03.817984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:03.818344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:03.818676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] PATH_INFO: `/v3/users/f4b71a4e5c1a4656a541f9cacec0af18` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:03.819807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b71a4e5c1a4656a541f9cacec0af18)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:03.821820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:03.876726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b71a4e5c1a4656a541f9cacec0af18'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:03.893043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b71a4e5c1a4656a541f9cacec0af18'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:03.900961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b71a4e5c1a4656a541f9cacec0af18'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:03.910184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b71a4e5c1a4656a541f9cacec0af18'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:03.911021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b71a4e5c1a4656a541f9cacec0af18'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:03.921629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1148/2305] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:00:03 2022] DELETE /identity/v3/users/f4b71a4e5c1a4656a541f9cacec0af18 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:00:04.107375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:04.118881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa174c8c-4f5d-4a2a-91f0-e387944d75b9 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:04.194001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:04.194679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:04.194960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:04.195237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] PATH_INFO: `/v3/projects/ad01763b3d76492598530234d3b75ec8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:04.196369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad01763b3d76492598530234d3b75ec8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:04.198310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:04.243774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad01763b3d76492598530234d3b75ec8', 'request_id': 'req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:04.249937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad01763b3d76492598530234d3b75ec8', 'request_id': 'req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:04.256251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad01763b3d76492598530234d3b75ec8', 'request_id': 'req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:04.268128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] The token cache is being invalidate because project ad01763b3d76492598530234d3b75ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:00:04.268418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:04.269037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c96847d-18b8-4572-b3ab-e3cacd94c6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:00:04.270523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1158/2306] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:00:04 2022] DELETE /identity/v3/projects/ad01763b3d76492598530234d3b75ec8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:00:13.914050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:00:14.265260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:00:28.348615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:28.350112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:28.427442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7497e6-85af-48fa-a696-3dd059c9d572 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:28.489319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:28.489926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:28.490212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:28.490485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:28.492332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:28.549077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:28.550719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:28.559893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1149/2307] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:38.555541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=84 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:00:40.814215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:40.814772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:40.814957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:40.836012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] MFA Rules not processed for user `6e3c1f7f6f8649a991d5d8a15eb8826b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:00:40.837541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:40.900552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1159/2308] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:00:41.418821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:41.420307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:41.482032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc4ff-97d1-4df6-9f63-4a4acb84cb46 service nova] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:41.485626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:41.486242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:41.486533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:41.486816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:41.488549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:41.499335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:41.500797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:41.510531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1150/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:41.654970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:41.656573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:41.674400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3b0aa-b413-45f4-a78a-f828bd38954b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:41.680327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:41.680970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:41.681930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:41.682278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:41.684422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:41.750291 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:41.752204 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd717122-dfdd-4178-b8b5-b8b7ece53400 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:41.764575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1160/2310] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:42.152313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:42.154213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:42.245255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4143b-0438-4633-8057-d5b2af7b2270 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:42.250347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:42.250988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:42.251266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:42.251538 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:42.255444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:42.267918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:42.269518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:42.282987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1151/2311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:43.409276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4fe5-3fae-48dc-8e1a-a0d45ff937bf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:43.409833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4fe5-3fae-48dc-8e1a-a0d45ff937bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:43.410022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4fe5-3fae-48dc-8e1a-a0d45ff937bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:43.434970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-422d4fe5-3fae-48dc-8e1a-a0d45ff937bf None None] MFA Rules not processed for user `f58a59c389cf4b3198a6c452eb2772d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:00:43.436293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-422d4fe5-3fae-48dc-8e1a-a0d45ff937bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:43.443473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1161/2312] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:00:43.759972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:00:43.761347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:43.772987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51aaa5-563b-4c4f-bf30-965830d45e4f service placement] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:00:43.777287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:00:43.777890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:00:43.778169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:00:43.778446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:00:43.780189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:00:43.789903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:00:43.791834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:00:43.801464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1152/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:00:53.428819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=56 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:00:53.797482 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=64 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:09.622622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:09.623130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:09.623225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:09.653227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] MFA Rules not processed for user `fb4bb9cd41cd46ddb87a746e732a948f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:09.654444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:09.725315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1162/2314] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:09.736471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:09.791671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef623a68-22b5-466c-bd19-05a16466665a service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:09.797611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:09.798269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:09.798592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:09.798857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:09.800582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:09.812161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:09.813733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:09.823611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1153/2315] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:15.870153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:15.894197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e3d93-5c4a-4d3e-b98e-7452098adfac None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:15.901225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:15.901534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:15.901928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:15.902209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:15.904243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:15.931031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:15.933209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:15.943967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1163/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:01:19.818666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=24 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:25.939182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=52 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:28.852815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:28.871971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d55be3-23ab-4a43-8db3-e672d4f4017e service nova] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:28.876861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:28.877646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:28.877921 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:28.878190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:28.880072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:28.963483 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:28.964948 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:28.984191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1154/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:32.473296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:32.474837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:32.488136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dae0a-1ff8-4745-b7ab-cc2109375dca service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:32.493889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:32.494636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:32.494981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:32.495324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:32.497154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:32.510576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:32.512696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:32.529416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1164/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:32.537643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:32.539780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:32.561128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be38d4-d97a-45b0-93c4-89f81cd7c29d service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:32.566277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:32.566989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:32.567401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:32.567852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] PATH_INFO: `/v3/users/66682d5656d3414eaebba666d4e0771f` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:32.568987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66682d5656d3414eaebba666d4e0771f)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:32.572817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:32.619620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66682d5656d3414eaebba666d4e0771f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.633056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66682d5656d3414eaebba666d4e0771f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.640494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66682d5656d3414eaebba666d4e0771f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.648075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66682d5656d3414eaebba666d4e0771f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.648731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0f4ea4e4-f476-4b05-92a1-4c35f7430979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66682d5656d3414eaebba666d4e0771f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.654521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1155/2319] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:32 2022] DELETE /identity/v3/users/66682d5656d3414eaebba666d4e0771f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:01:32.826460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:32.838274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e687-1454-4764-bbbb-82e8db74a3b8 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:32.894994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:32.895627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:32.896053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:32.896336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] PATH_INFO: `/v3/projects/92efab98bce54b08899f057951b42617` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:32.898130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92efab98bce54b08899f057951b42617)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:32.900112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:32.946337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92efab98bce54b08899f057951b42617', 'request_id': 'req-9092bdc4-72d4-4807-af1c-4a1dc5426b39'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.952160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92efab98bce54b08899f057951b42617', 'request_id': 'req-9092bdc4-72d4-4807-af1c-4a1dc5426b39'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.958307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92efab98bce54b08899f057951b42617', 'request_id': 'req-9092bdc4-72d4-4807-af1c-4a1dc5426b39'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.972696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] The token cache is being invalidate because project 92efab98bce54b08899f057951b42617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:32.972984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.973629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9092bdc4-72d4-4807-af1c-4a1dc5426b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:32.975428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1165/2320] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:32 2022] DELETE /identity/v3/projects/92efab98bce54b08899f057951b42617 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:32.981000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:32.981305 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:32.981579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.020731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:33.040768 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:33.041961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:33.102865 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1156/2321] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:33.111671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.111969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.112226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.134207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:33.135483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:33.143472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1166/2322] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:33.147854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.160301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b441b-074e-4ce7-8aa1-4232ba8b2910 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.166237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.167337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.167815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.168283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.169022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:33.169611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.172238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.190559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1157/2323] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:01:33.194964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.204609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30982a-f4ba-4264-88ae-3696e5c4a0f5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.211878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.212735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.213172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.213617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.214370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.216547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.247383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1167/2324] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:33 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:33.252413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.263496 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13808cf-d027-48a6-ad7c-17be50ebcc4c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.343674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.344289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.344570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.344842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.345490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.347470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.370977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1158/2325] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:33 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:33.374989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.385797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8ee74-7d54-4c1d-8264-f74323c6638f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.390565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.391186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.391466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.391739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.392394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.393995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.405110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1168/2326] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:33.409142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.419861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad0e9e-24f9-4944-80e6-11c9e992b52d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.424644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.425333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.425615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.425919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.426621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.428388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd413d6-08f2-4878-9070-ecb0ea479d09 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.439630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1159/2327] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:33.443703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.454526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207a3e-b3bd-4318-a4a6-375ead4aa9d0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.459222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.459899 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.460839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.461122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] PATH_INFO: `/v3/projects/e2f949fe717f488c8ff3ce5ac3145fe2/users/83c4ec497c1f41619f6da313ebbc25e3/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.476382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f949fe717f488c8ff3ce5ac3145fe2, user_id=83c4ec497c1f41619f6da313ebbc25e3, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.479332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.494936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1169/2328] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:33 2022] PUT /identity/v3/projects/e2f949fe717f488c8ff3ce5ac3145fe2/users/83c4ec497c1f41619f6da313ebbc25e3/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:01:33.505917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.506365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.507298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.536718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] MFA Rules not processed for user `83c4ec497c1f41619f6da313ebbc25e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:33.537984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:33.595346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:33.607117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1160/2329] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:33.617656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:33.619370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:33.644635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:33.718799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343ab1f-e585-4d57-a57b-74393655c664 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:33.724402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:33.725013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:33.725300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:33.725600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:33.733780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:33.735404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd2584-eb54-47ec-bb01-7f309ee91a5d service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:33.745155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1170/2330] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:36.487845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:36.546971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd48c-b962-4b82-8976-43b102a7dd0a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:36.551470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:36.552113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:36.552390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:36.552660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] PATH_INFO: `/v3/users/83c4ec497c1f41619f6da313ebbc25e3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:36.553949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c4ec497c1f41619f6da313ebbc25e3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:36.556154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:36.604195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c4ec497c1f41619f6da313ebbc25e3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:36.616053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c4ec497c1f41619f6da313ebbc25e3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:36.623160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c4ec497c1f41619f6da313ebbc25e3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:36.630380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c4ec497c1f41619f6da313ebbc25e3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:36.631137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c4ec497c1f41619f6da313ebbc25e3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:36.637725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1161/2331] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:36 2022] DELETE /identity/v3/users/83c4ec497c1f41619f6da313ebbc25e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:36.642365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:36.642892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:36.643006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:36.662320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:36.663630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:36.722370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1171/2332] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:36.732074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:36.733465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:36.801195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a7d3f-ef2f-4262-958d-01e0ceffa2ef admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:36.806121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:36.806762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:36.807040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:36.807307 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:36.816132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:36.817554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dcd1f-3ed7-4d4f-bb09-7bff66debf11 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:36.827605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1162/2333] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:37.095311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.096759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:37.109568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75367c-6bf3-4bdc-9c7b-a11cf487ee58 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.114886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.115606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.116504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.116795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] PATH_INFO: `/v3/projects/e2f949fe717f488c8ff3ce5ac3145fe2` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.118010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f949fe717f488c8ff3ce5ac3145fe2)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.119975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.172476 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f949fe717f488c8ff3ce5ac3145fe2', 'request_id': 'req-a0db71f7-1a5b-4064-b0f1-1a477c23c682'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:37.178488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f949fe717f488c8ff3ce5ac3145fe2', 'request_id': 'req-a0db71f7-1a5b-4064-b0f1-1a477c23c682'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:37.185287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f949fe717f488c8ff3ce5ac3145fe2', 'request_id': 'req-a0db71f7-1a5b-4064-b0f1-1a477c23c682'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:37.199283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] The token cache is being invalidate because project e2f949fe717f488c8ff3ce5ac3145fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:37.199572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:37.200220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a0db71f7-1a5b-4064-b0f1-1a477c23c682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:37.201746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1172/2334] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:37 2022] DELETE /identity/v3/projects/e2f949fe717f488c8ff3ce5ac3145fe2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:01:37.207604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215899c-2f70-4292-a801-8de820334c16 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.207900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215899c-2f70-4292-a801-8de820334c16 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.208165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215899c-2f70-4292-a801-8de820334c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.218573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:37.231133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3215899c-2f70-4292-a801-8de820334c16 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:37.232940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3215899c-2f70-4292-a801-8de820334c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:37.296736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1163/2335] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:37.305568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.305892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.306165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.326259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:37.327547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:37.335807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1173/2336] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:37.339925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215899c-2f70-4292-a801-8de820334c16 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.349619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215899c-2f70-4292-a801-8de820334c16 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.354784 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.355386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.355670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.355943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.356586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:37.356895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.358493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.370549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1164/2337] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:01:37.374694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.380627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:37.384373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baa487-39d9-401d-9418-01bf794d4a6d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.390412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.391132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.391473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.391932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.392579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.394588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.421883 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1174/2338] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:37.425949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.436674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3121a-6bec-4435-ad1e-a3ec1cc381d8 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.494208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.494846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.495088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.495417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.496059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.498021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.520514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1165/2339] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:37.525208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.535987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be435e2-633b-4718-b84e-3cf250391331 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.541129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.541720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.541999 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.542268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.542959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.544553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.554996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1175/2340] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:37.559302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.569262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94438-3d40-4b71-bc7d-c4fbfa41136a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.574001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.574625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.574906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.575180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.575822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.577407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84fc6-bfd3-40f4-9e3c-964dda0bd25d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.588512 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1166/2341] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:37.592506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.602965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932095d-22d2-4391-9aea-1caf18311a7c admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.607505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.608136 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.608412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.608680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] PATH_INFO: `/v3/projects/5b628ac67c944ed48693f5ef7a27ef49/users/d957bc845f3c43aeb7c705caf2367d4f/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.622820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b628ac67c944ed48693f5ef7a27ef49, user_id=d957bc845f3c43aeb7c705caf2367d4f, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.625446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.637992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1176/2342] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:37 2022] PUT /identity/v3/projects/5b628ac67c944ed48693f5ef7a27ef49/users/d957bc845f3c43aeb7c705caf2367d4f/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:37.649962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.650382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.650811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.679377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] MFA Rules not processed for user `d957bc845f3c43aeb7c705caf2367d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:37.680855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:37.742572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:37.746984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1167/2343] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:37.751282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.813526 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9226d41-cd75-4a23-b6d6-d32997dcfd92 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.818078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.818774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.819055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.819328 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.819949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.821857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.848212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1177/2344] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:37.853036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.864088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa6872-ffc9-4826-a2d5-d4c1e048f1c6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.920084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.920695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.920979 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.921257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.921879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.923861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.945147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1168/2345] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:37 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:37.949255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.960168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4063a-2b0e-49fb-98eb-2ae586a44373 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.964729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.965352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.965663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.965937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:37.966618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:37.968209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:37.978854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1178/2346] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:37.982955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:37.993060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d58f0-584b-468b-9e64-cf071f071f02 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:37.997541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:37.998136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:37.998420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:37.998755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] PATH_INFO: `/v3/projects/31a7da831a674ada8c287d2f175aab75/users/9a304e4272a84458b2979eaf6260cab1/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.013772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a7da831a674ada8c287d2f175aab75, user_id=9a304e4272a84458b2979eaf6260cab1, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:38.016503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:38.030301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1169/2347] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:37 2022] PUT /identity/v3/projects/31a7da831a674ada8c287d2f175aab75/users/9a304e4272a84458b2979eaf6260cab1/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:38.034277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:38.044814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bfd5-4cc1-42d2-aef4-d686b316ae06 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:38.100797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:38.101401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:38.101680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:38.101955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.102633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:38.104260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef9ba-9e36-4e93-b039-fba94d2f6762 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:38.114592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1179/2348] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:38.125771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:38.136477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af242f-3de3-4455-93f6-e23ec910b1fd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:38.141046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:38.141685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:38.141963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:38.142231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] PATH_INFO: `/v3/projects/31a7da831a674ada8c287d2f175aab75/users/9a304e4272a84458b2979eaf6260cab1/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.144144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a7da831a674ada8c287d2f175aab75, user_id=9a304e4272a84458b2979eaf6260cab1, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:38.146779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:38.157606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1170/2349] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:38 2022] PUT /identity/v3/projects/31a7da831a674ada8c287d2f175aab75/users/9a304e4272a84458b2979eaf6260cab1/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:01:38.167812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:38.168096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:38.168369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.196278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] MFA Rules not processed for user `9a304e4272a84458b2979eaf6260cab1`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:38.197474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:38.202360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:38.267053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1180/2350] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:38.278608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:38.280322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:38.357908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fa2b9-4e3c-4d8a-981a-045a595ca707 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:38.361532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:38.362134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:38.362409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:38.362713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.372287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:38.373779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:38.383800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1171/2351] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:38.428853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:38.430671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:38.442805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cecfc-aef0-46e5-8e21-bb5fc0532409 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:38.447443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:38.448070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:38.448347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:38.448615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:38.504906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:38.506895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3651a-94d2-4438-85d0-2f07372533d4 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:38.515884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1181/2352] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:01:41.795025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:41.836779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:41.860928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adec3d-2a0e-442e-8f02-8b73a45a74ae service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:41.865360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:41.866671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:41.866951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:41.867219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] PATH_INFO: `/v3/users/d957bc845f3c43aeb7c705caf2367d4f` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:41.868319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d957bc845f3c43aeb7c705caf2367d4f)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:41.870217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:41.916007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd957bc845f3c43aeb7c705caf2367d4f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:41.927508 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd957bc845f3c43aeb7c705caf2367d4f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:41.935600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd957bc845f3c43aeb7c705caf2367d4f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:41.942471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd957bc845f3c43aeb7c705caf2367d4f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:41.943142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd957bc845f3c43aeb7c705caf2367d4f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:41.949244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1172/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:41 2022] DELETE /identity/v3/users/d957bc845f3c43aeb7c705caf2367d4f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:41.953689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:41.954178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:41.954275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:41.973075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:41.974310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:42.039249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1182/2354] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:42.048805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:42.050368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:42.120365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87252-d19d-4430-a0a7-9255b1aa0cf5 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:42.124193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:42.124912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:42.125256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:42.125535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:42.135059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:42.136517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:42.146064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1173/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:42.416880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:42.418376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:42.436416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfee26-ac8e-4549-b66e-75be8fa7f4d1 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:42.443609 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:42.444689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:42.445182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:42.445671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] PATH_INFO: `/v3/projects/5b628ac67c944ed48693f5ef7a27ef49` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:42.447585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b628ac67c944ed48693f5ef7a27ef49)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:42.451323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:42.489520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:42.499259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b628ac67c944ed48693f5ef7a27ef49', 'request_id': 'req-84591fc6-3a40-4b48-916e-912669dfb8a0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.504799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b628ac67c944ed48693f5ef7a27ef49', 'request_id': 'req-84591fc6-3a40-4b48-916e-912669dfb8a0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.511308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b628ac67c944ed48693f5ef7a27ef49', 'request_id': 'req-84591fc6-3a40-4b48-916e-912669dfb8a0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.523336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] The token cache is being invalidate because project 5b628ac67c944ed48693f5ef7a27ef49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:42.523623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.524232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.525845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1183/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:42 2022] DELETE /identity/v3/projects/5b628ac67c944ed48693f5ef7a27ef49 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:42.529781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:42.540447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4f52-6982-4e87-82e8-6870e6cdf7bc service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:42.590378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:42.596990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:42.597676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:42.598122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:42.598618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] PATH_INFO: `/v3/users/9a304e4272a84458b2979eaf6260cab1` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:42.600438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a304e4272a84458b2979eaf6260cab1)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:42.602584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:42.648289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a304e4272a84458b2979eaf6260cab1'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.659991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a304e4272a84458b2979eaf6260cab1'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.668008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a304e4272a84458b2979eaf6260cab1'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.675003 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a304e4272a84458b2979eaf6260cab1'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.675626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bcbe89eb-6068-4a28-ba5a-deb3c6f95c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a304e4272a84458b2979eaf6260cab1'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:42.682276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1174/2357] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:42 2022] DELETE /identity/v3/users/9a304e4272a84458b2979eaf6260cab1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:42.950794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:42.952311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:42.965541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84591fc6-3a40-4b48-916e-912669dfb8a0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.029365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.030000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.030278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.030580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] PATH_INFO: `/v3/projects/31a7da831a674ada8c287d2f175aab75` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.031719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a7da831a674ada8c287d2f175aab75)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.033662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.079781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a7da831a674ada8c287d2f175aab75', 'request_id': 'req-edefbe99-34db-454e-94a0-1c62def6136c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:43.085277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a7da831a674ada8c287d2f175aab75', 'request_id': 'req-edefbe99-34db-454e-94a0-1c62def6136c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:43.091415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a7da831a674ada8c287d2f175aab75', 'request_id': 'req-edefbe99-34db-454e-94a0-1c62def6136c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:43.104198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] The token cache is being invalidate because project 31a7da831a674ada8c287d2f175aab75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:43.104483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:43.105095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-edefbe99-34db-454e-94a0-1c62def6136c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:43.106686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1184/2358] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:42 2022] DELETE /identity/v3/projects/31a7da831a674ada8c287d2f175aab75 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:43.111739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.112213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.112426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.134927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:43.136137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:43.194973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1175/2359] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:43.204214 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.204596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.204878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.225029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:43.226233 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:43.234000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1185/2360] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:43.238225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.248113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff213162-7484-4e67-8b28-11db37bc292b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.252670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.253271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.253552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.253841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.254478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:43.254816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.256950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.268415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1176/2361] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:01:43.273276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.282560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caba30-5e40-45a2-8762-570c28a7a46a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.287571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.288336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.288617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.288929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.289572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.291556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.319796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1186/2362] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:43.324053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.337183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c85f-5ff3-45e4-80da-ac8f05fb4a25 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.393287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.393889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.394168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.394439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.395092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.397818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.407486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:43.420219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1177/2363] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:43.424354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.434798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bdba3-98de-4ca4-9789-12fab7d2a8b0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.439411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.440002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.440282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.440554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.441194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.442824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.453359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1187/2364] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:43.457497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.468174 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35616-1b3a-4db5-a540-ab40f8bcc901 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.472646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.473233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.473534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.473805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.474437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.476191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c464-bf9e-4fc4-a77a-450929955498 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.486632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1178/2365] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:43.491252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.501380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a82b-e35c-4b07-b8dd-2ea1e86c52f2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.506688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.507479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.507761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.508039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] PATH_INFO: `/v3/projects/735306e182df4167912e595df24d826d/users/537f4bee34cb4664a305a3a2efdf510e/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.519938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:43.523168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] RBAC: Authorizing `identity:create_grant(project_id=735306e182df4167912e595df24d826d, user_id=537f4bee34cb4664a305a3a2efdf510e, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.526549 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.539541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1188/2366] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:43 2022] PUT /identity/v3/projects/735306e182df4167912e595df24d826d/users/537f4bee34cb4664a305a3a2efdf510e/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:43.549928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.550219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.550494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.578928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] MFA Rules not processed for user `537f4bee34cb4664a305a3a2efdf510e`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:43.580141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:43.642412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1179/2367] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:43.646984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.708333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf55f5a-bfcc-4bf7-9200-3aa78e25000d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.713197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.713799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.714077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.714351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.715013 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.716911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.742925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1189/2368] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:43.746977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.758298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26179401-810b-4bdc-bf41-a8bbe133b733 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.813729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.814334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.814657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.814933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.815557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.817476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.837862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1180/2369] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:43 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:43.841789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.852369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a935a-d28a-41c3-907c-247242bc51e3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.856916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.857503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.857779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.858049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.858734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.860318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.870703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1190/2370] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:43.874635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.884819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3796-33ec-47d8-a15d-2bd852ebbc3a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.889301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.889937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.890215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.890481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] PATH_INFO: `/v3/projects/2aceeb750a7c46fbaccf67d690157af2/users/d25b0d655e3c42019a5dc287757b092e/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.905806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aceeb750a7c46fbaccf67d690157af2, user_id=d25b0d655e3c42019a5dc287757b092e, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.908446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:43.921899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1181/2371] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:43 2022] PUT /identity/v3/projects/2aceeb750a7c46fbaccf67d690157af2/users/d25b0d655e3c42019a5dc287757b092e/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:01:43.925981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:43.937024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90cb03-35ea-4774-9a6b-4bafd0c919df admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:43.993262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:43.993943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:43.994132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:43.994403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:43.995078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:43.996727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477b146-df7f-4d4d-aa8b-3afd9bbd5e53 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.006462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1191/2372] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:44.010654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.020690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7eeede-93e6-43df-8801-bbddeeccf383 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.025281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:44.025939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.026216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.026480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] PATH_INFO: `/v3/projects/2aceeb750a7c46fbaccf67d690157af2/users/d25b0d655e3c42019a5dc287757b092e/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.028466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aceeb750a7c46fbaccf67d690157af2, user_id=d25b0d655e3c42019a5dc287757b092e, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:44.031147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.041695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1182/2373] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:44 2022] PUT /identity/v3/projects/2aceeb750a7c46fbaccf67d690157af2/users/d25b0d655e3c42019a5dc287757b092e/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:44.051725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.052012 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.052286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.081139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] MFA Rules not processed for user `d25b0d655e3c42019a5dc287757b092e`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:44.082347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:44.131986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:44.153730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1192/2374] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:44.165454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.167200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:44.194975 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:44.226033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23f250-a006-441d-8788-5646a7789eb3 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.229658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:44.230261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.230559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.230830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.239683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:44.241722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732cb71-7e8a-451e-9df1-89fe0dfb9c8c service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.252880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1183/2375] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:44.282544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.345904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6c2f7-b1e6-4234-90ff-86999d733b5f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.351498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:44.352122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.352407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.352684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] PATH_INFO: `/v3/users/537f4bee34cb4664a305a3a2efdf510e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.353826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537f4bee34cb4664a305a3a2efdf510e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:44.355780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.401798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537f4bee34cb4664a305a3a2efdf510e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.410582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537f4bee34cb4664a305a3a2efdf510e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.426215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537f4bee34cb4664a305a3a2efdf510e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.427056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537f4bee34cb4664a305a3a2efdf510e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.434715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537f4bee34cb4664a305a3a2efdf510e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.441251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1193/2376] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:44 2022] DELETE /identity/v3/users/537f4bee34cb4664a305a3a2efdf510e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:44.445857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.446145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.446448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.465754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:44.466991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:44.524968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1184/2377] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:44.534548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.535947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:44.593063 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b576-c763-4ea9-97c2-82bc80b5fc18 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.596631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:44.597224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.597501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.597775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.607358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:44.608767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.618122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1194/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:44.876501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.878032 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:44.891290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b72fc-75b9-4576-b440-49d0387eff8c None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.895950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:44.897272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:44.897605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:44.897928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] PATH_INFO: `/v3/projects/735306e182df4167912e595df24d826d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:44.899085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=735306e182df4167912e595df24d826d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:44.901006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:44.947132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735306e182df4167912e595df24d826d', 'request_id': 'req-e8294482-c740-44f0-9add-c11a987797bd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.955186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735306e182df4167912e595df24d826d', 'request_id': 'req-e8294482-c740-44f0-9add-c11a987797bd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.961317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735306e182df4167912e595df24d826d', 'request_id': 'req-e8294482-c740-44f0-9add-c11a987797bd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.972325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] The token cache is being invalidate because project 735306e182df4167912e595df24d826d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:44.972610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.973263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:44.975368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1185/2379] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:44 2022] DELETE /identity/v3/projects/735306e182df4167912e595df24d826d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:44.979338 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:44.988879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2483b4-696f-43da-9a2e-6a1d5dc28d31 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:44.994182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:45.046946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.047602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.047891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.048166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] PATH_INFO: `/v3/users/d25b0d655e3c42019a5dc287757b092e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.049306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25b0d655e3c42019a5dc287757b092e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.051506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.097651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25b0d655e3c42019a5dc287757b092e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.106425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25b0d655e3c42019a5dc287757b092e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.118931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25b0d655e3c42019a5dc287757b092e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.119644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25b0d655e3c42019a5dc287757b092e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.126408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f70abf47-de72-4295-9422-92e1ec248400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25b0d655e3c42019a5dc287757b092e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.132227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1195/2380] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:44 2022] DELETE /identity/v3/users/d25b0d655e3c42019a5dc287757b092e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:45.400164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.401954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:45.420242 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8294482-c740-44f0-9add-c11a987797bd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.469488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:45.480933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.481498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.481774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.482044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] PATH_INFO: `/v3/projects/2aceeb750a7c46fbaccf67d690157af2` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.483158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aceeb750a7c46fbaccf67d690157af2)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.485107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.531600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aceeb750a7c46fbaccf67d690157af2', 'request_id': 'req-fb2da307-b47b-416b-81ee-371ada25ce1e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.538954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aceeb750a7c46fbaccf67d690157af2', 'request_id': 'req-fb2da307-b47b-416b-81ee-371ada25ce1e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.544549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aceeb750a7c46fbaccf67d690157af2', 'request_id': 'req-fb2da307-b47b-416b-81ee-371ada25ce1e'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.555751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] The token cache is being invalidate because project 2aceeb750a7c46fbaccf67d690157af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:45.556034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.556658 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-fb2da307-b47b-416b-81ee-371ada25ce1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:45.558129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1186/2381] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:45 2022] DELETE /identity/v3/projects/2aceeb750a7c46fbaccf67d690157af2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:45.563093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.563385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.563666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.583617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:45.585486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:45.643662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1196/2382] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:45.653092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.653373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.653643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.673187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:45.674407 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:45.682572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1187/2383] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:45.686704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.696799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075a639-425e-4d1e-8d7f-b7dcc34e7a02 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.701284 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.701874 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.702151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.702422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.703091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:45.703395 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.704977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.716955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1197/2384] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:01:45.721231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.730461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac547189-0ef9-495c-a6da-f1a604b34600 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.735129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.735656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.735955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.736224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.736850 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.738757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.765721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1188/2385] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:45 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:45.769740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.779490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a462b04-0d43-46db-8a62-1778d67a040a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.791041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:45.839055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.839659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.839940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.840213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.840847 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.842770 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.863782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1198/2386] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:45 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:45.868342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.878748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522937bf-9755-4442-88b3-67acbe038309 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.883981 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.884581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.884860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.885130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.885770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.887401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.897308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1189/2387] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:45.901043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.910758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cf349-7073-4b99-b62d-bf56ebaa91c0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.915433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.916028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.916308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.916584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.917227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.918837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c956cb5-33d3-4ffa-b887-2156e67918ad admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.929793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1199/2388] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:45.933894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:45.944726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b196bd4-d9f4-4149-b934-693d8c25a633 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:45.949451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:45.950331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.950798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.951075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] PATH_INFO: `/v3/projects/10e4d9e7986640b4a39be6644587c140/users/a867210413e045d2b5c1bf3f9aa3eb7c/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:45.965812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e4d9e7986640b4a39be6644587c140, user_id=a867210413e045d2b5c1bf3f9aa3eb7c, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:45.968485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:45.981778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1190/2389] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:45 2022] PUT /identity/v3/projects/10e4d9e7986640b4a39be6644587c140/users/a867210413e045d2b5c1bf3f9aa3eb7c/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:45.992846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:45.993078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:45.993358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.021458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] MFA Rules not processed for user `a867210413e045d2b5c1bf3f9aa3eb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:46.022993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:46.088401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1200/2390] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:46.092636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.153994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28cb7a-3282-4b18-8cec-7afa482c8031 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.158690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.159294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.159576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.159854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.160474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.163179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.189408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1191/2391] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:46 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:46.193444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.204077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77226012-abf9-429c-bc82-a0bd5981f45a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.260683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.261217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.261498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.261772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.262409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.264385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.285959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1201/2392] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:46 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:46.290530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.295957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:46.302393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4a413-c584-4e41-b6f5-25257ed17da8 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.308930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.309523 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.309803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.310074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.310745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.312330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.322785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1192/2393] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:46.326807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.337043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c8c3a-d35f-4322-8619-586356c9bcf5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.341850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.342484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.342805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.343075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] PATH_INFO: `/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4/users/47eb2bcf4d6f4633b2fcdbd393221bca/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.356448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:46.361104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6a82a9abf2411b84388ff6a827c8a4, user_id=47eb2bcf4d6f4633b2fcdbd393221bca, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.364138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.376882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1202/2394] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:46 2022] PUT /identity/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4/users/47eb2bcf4d6f4633b2fcdbd393221bca/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:46.381329 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.391381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0938-c21d-4a26-b86f-fd9bea802447 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.453022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.453638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.453895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.454166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.454847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.456441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96a7e2-0d1f-4a23-b036-57b2f0fcfe58 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.466623 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1193/2395] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:01:46.470693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.481461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bfe3-0e7c-4198-b5d7-e82bfcc4f277 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.486076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.486750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.487030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.487306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] PATH_INFO: `/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4/users/47eb2bcf4d6f4633b2fcdbd393221bca/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.489185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6a82a9abf2411b84388ff6a827c8a4, user_id=47eb2bcf4d6f4633b2fcdbd393221bca, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.492455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.502392 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1203/2396] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:46 2022] PUT /identity/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4/users/47eb2bcf4d6f4633b2fcdbd393221bca/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:46.513643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.513870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.514145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.542144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] MFA Rules not processed for user `47eb2bcf4d6f4633b2fcdbd393221bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:46.543386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:46.610304 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1194/2397] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:46.621631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.623370 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:46.680352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdac7-9d02-4a78-9a33-aebf1e19a6c3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.683946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.684539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.684815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.685084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.699729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.701158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.711083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1204/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:46.792311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:46.802475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b4c64-37b8-4787-bc4e-279bb7ed38cd None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:46.806587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:46.807182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:46.807458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:46.807725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:46.863968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:46.865457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b6db-84cd-49f8-bee5-5cb589a49108 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:46.874297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1195/2399] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:51.575777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:51.636718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9abd-e61d-477c-8979-2b19ca19acb4 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:51.641211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:51.641825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:51.642103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:51.642373 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] PATH_INFO: `/v3/users/a867210413e045d2b5c1bf3f9aa3eb7c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:51.643518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a867210413e045d2b5c1bf3f9aa3eb7c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:51.645401 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:51.692112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a867210413e045d2b5c1bf3f9aa3eb7c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:51.700315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a867210413e045d2b5c1bf3f9aa3eb7c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:51.713264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a867210413e045d2b5c1bf3f9aa3eb7c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:51.713924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a867210413e045d2b5c1bf3f9aa3eb7c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:51.720694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a867210413e045d2b5c1bf3f9aa3eb7c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:51.722188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:51.727885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1205/2400] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:51 2022] DELETE /identity/v3/users/a867210413e045d2b5c1bf3f9aa3eb7c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:51.732312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:51.732808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:51.732914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:51.753001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:51.754190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:51.759888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:51.814841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1196/2401] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:51.825544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:51.826974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:51.884146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2450d-5ebb-4c04-ba95-d245e156bbef admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:51.889229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:51.889854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:51.890133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:51.890550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:51.900488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:51.901922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:51.913283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1206/2402] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:01:52.183907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:52.185446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:52.198026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7920a5a-1e2a-4517-9e4e-632cbd9e5b44 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:52.202814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:52.203518 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:52.203866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:52.204203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] PATH_INFO: `/v3/projects/10e4d9e7986640b4a39be6644587c140` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:52.205389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e4d9e7986640b4a39be6644587c140)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:52.207444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:52.256726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e4d9e7986640b4a39be6644587c140', 'request_id': 'req-bc556643-b134-4116-b2b8-4b53ff378589'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.263211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e4d9e7986640b4a39be6644587c140', 'request_id': 'req-bc556643-b134-4116-b2b8-4b53ff378589'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.268886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e4d9e7986640b4a39be6644587c140', 'request_id': 'req-bc556643-b134-4116-b2b8-4b53ff378589'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.279859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] The token cache is being invalidate because project 10e4d9e7986640b4a39be6644587c140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:52.280255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.280916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.282492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1197/2403] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:52 2022] DELETE /identity/v3/projects/10e4d9e7986640b4a39be6644587c140 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:52.286481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:52.296567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177749bf-b526-4b3b-ba6e-8931069703a6 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:52.352276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:52.352907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:52.353184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:52.353453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] PATH_INFO: `/v3/users/47eb2bcf4d6f4633b2fcdbd393221bca` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:52.354593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47eb2bcf4d6f4633b2fcdbd393221bca)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:52.356574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:52.404006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47eb2bcf4d6f4633b2fcdbd393221bca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.411153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47eb2bcf4d6f4633b2fcdbd393221bca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.423371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47eb2bcf4d6f4633b2fcdbd393221bca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.424014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47eb2bcf4d6f4633b2fcdbd393221bca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.431487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6cd18a28-b58b-4b4b-904b-b0d5404b1dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47eb2bcf4d6f4633b2fcdbd393221bca'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.437397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1207/2404] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:52 2022] DELETE /identity/v3/users/47eb2bcf4d6f4633b2fcdbd393221bca => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:52.700914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:52.702437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:52.720716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc556643-b134-4116-b2b8-4b53ff378589 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:52.778086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:52.778763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:52.779015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:52.779288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] PATH_INFO: `/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:52.780457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6a82a9abf2411b84388ff6a827c8a4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:52.782317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:52.820761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:52.830445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6a82a9abf2411b84388ff6a827c8a4', 'request_id': 'req-476466b0-f22d-492e-9f8e-fa68e954552a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.837580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6a82a9abf2411b84388ff6a827c8a4', 'request_id': 'req-476466b0-f22d-492e-9f8e-fa68e954552a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.842996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6a82a9abf2411b84388ff6a827c8a4', 'request_id': 'req-476466b0-f22d-492e-9f8e-fa68e954552a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.854365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] The token cache is being invalidate because project 2a6a82a9abf2411b84388ff6a827c8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:52.854678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.855296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-476466b0-f22d-492e-9f8e-fa68e954552a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:52.856749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1198/2405] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:52 2022] DELETE /identity/v3/projects/2a6a82a9abf2411b84388ff6a827c8a4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:52.861691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:52.862206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:52.862303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:52.882589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:52.883800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:52.896503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:52.944045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1208/2406] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:52.951964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:52.952318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:52.952656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:52.972602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:52.973869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:52.982319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1199/2407] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:52.986458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:52.996845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc261195-264c-4692-9c08-a42bccad3779 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.001505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.002104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.002383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.002691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.003339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:53.003641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.005246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.016829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1209/2408] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:01:53.021080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.030199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193852c6-3447-40f1-89df-64fe3b216798 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.034887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.035424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.035705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.035978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.037301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.039238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.066204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1200/2409] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:53.070765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.080955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec770f1-34e4-4496-a218-f9b4eec1f243 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.140661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.141269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.141548 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.141824 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.142467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.144410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.165640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1210/2410] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:53.169753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.181074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67b3bc-8a54-4d88-97dd-575fc23e9681 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.185675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.186271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.186583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.186857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.187501 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.189095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.201395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1201/2411] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:53.205623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.215948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619297-0a35-431d-af55-63fa62268842 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.220610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.221201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.221481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.221756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.222407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.224543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a873110-2522-4390-bc0b-7e53f236f348 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.235505 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1211/2412] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:01:53.240266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.250183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbcbb-87f9-44ed-93ee-87f3d88ae03b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.255040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.255688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.255964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.256233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] PATH_INFO: `/v3/projects/2b5d923cac464199a07a9d3863160a48/users/56c388d33a354f84b36ac676f27c5337/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.270380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5d923cac464199a07a9d3863160a48, user_id=56c388d33a354f84b36ac676f27c5337, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.273662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.285906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1202/2413] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:53 2022] PUT /identity/v3/projects/2b5d923cac464199a07a9d3863160a48/users/56c388d33a354f84b36ac676f27c5337/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:53.296719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.297004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.297282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.326175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] MFA Rules not processed for user `56c388d33a354f84b36ac676f27c5337`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:53.327413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:53.395162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1212/2414] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:53.400347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.461051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f2d8c-736a-4dfd-9340-ae1c91a965ee admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.465643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.466236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.466588 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.466816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.467442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.469338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.496070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1203/2415] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:53.500121 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.510878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52606fe1-341d-4cc0-abb6-9423d9bcf648 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.522744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:53.569798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.570335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.570646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.570920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.571571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.573545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.594886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1213/2416] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:53.598846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.609596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0c86-1d2d-4a3f-a566-1e2729a6c00b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.614112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.615361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.615641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.615909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.616546 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.618136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.628402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1204/2417] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:53.632457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.642389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d7dda-5a75-4c5c-834e-feeda586541a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.647109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.647749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.648030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.648304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] PATH_INFO: `/v3/projects/fef953b584de44a994565aed2c8fd5ea/users/2736ac24707942448667e4e1913aa8b8/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.663698 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef953b584de44a994565aed2c8fd5ea, user_id=2736ac24707942448667e4e1913aa8b8, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.666317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.679016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1214/2418] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:53 2022] PUT /identity/v3/projects/fef953b584de44a994565aed2c8fd5ea/users/2736ac24707942448667e4e1913aa8b8/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:53.683154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.692986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0ce42-f0b6-4cc8-bb8c-50b40e245bd4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.726019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:53.751683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.752285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.752567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.752841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.753485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.755127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada3c86-39c9-4da8-8f1f-bf9e9f80e58c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.764806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1205/2419] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:53.768838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.778947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f0f1a-0d9b-4486-a838-efc04a76d437 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:53.783502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:53.784145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.784424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.784697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] PATH_INFO: `/v3/projects/fef953b584de44a994565aed2c8fd5ea/users/2736ac24707942448667e4e1913aa8b8/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.787198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef953b584de44a994565aed2c8fd5ea, user_id=2736ac24707942448667e4e1913aa8b8, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:53.789795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:53.799924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1215/2420] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:53 2022] PUT /identity/v3/projects/fef953b584de44a994565aed2c8fd5ea/users/2736ac24707942448667e4e1913aa8b8/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:53.865112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:53.865704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:53.865704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:53.893437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] MFA Rules not processed for user `2736ac24707942448667e4e1913aa8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:53.894680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:53.967115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1206/2421] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:53.978954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:53.980713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:54.039164 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61110-6ea0-4827-9e57-b8df8950457d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:54.042833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:54.043437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:54.043769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:54.044046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:54.053116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:54.054601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd3000-191a-4d3c-b59c-a2fda4148a3c service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:54.064348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1216/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:01:54.437217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:54.500414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce283-8bff-4f98-a40a-947cb6ca4b6e None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:54.504934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:54.505599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:54.505881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:54.506158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] PATH_INFO: `/v3/users/56c388d33a354f84b36ac676f27c5337` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:54.507312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c388d33a354f84b36ac676f27c5337)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:54.509240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:54.555781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c388d33a354f84b36ac676f27c5337'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:54.563320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:54.568820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c388d33a354f84b36ac676f27c5337'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:54.576221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c388d33a354f84b36ac676f27c5337'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:54.583137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c388d33a354f84b36ac676f27c5337'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:54.583804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c388d33a354f84b36ac676f27c5337'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:54.589611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1207/2423] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:54 2022] DELETE /identity/v3/users/56c388d33a354f84b36ac676f27c5337 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:54.594189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:54.594475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:54.594781 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:54.613907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:54.615227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:54.672513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1217/2424] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:54.682415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:54.683899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:54.740394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4e021-b84a-4552-8c59-3de6cf1fa3e8 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:54.744033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:54.744627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:54.744907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:54.745176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:54.754037 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:54.756184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:54.765564 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1208/2425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:01:55.039105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.040624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:55.055091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a23594-84c0-4343-978a-4b843e5a86b3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:55.059998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:55.060673 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.061071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.061350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] PATH_INFO: `/v3/projects/2b5d923cac464199a07a9d3863160a48` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.062483 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5d923cac464199a07a9d3863160a48)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:55.064478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:55.072047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:55.112869 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5d923cac464199a07a9d3863160a48', 'request_id': 'req-3b545b5d-8916-4cd1-820d-1383636fcdda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.119568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5d923cac464199a07a9d3863160a48', 'request_id': 'req-3b545b5d-8916-4cd1-820d-1383636fcdda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.125917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5d923cac464199a07a9d3863160a48', 'request_id': 'req-3b545b5d-8916-4cd1-820d-1383636fcdda'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.139677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] The token cache is being invalidate because project 2b5d923cac464199a07a9d3863160a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:55.139965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.140580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.142068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1218/2426] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:55 2022] DELETE /identity/v3/projects/2b5d923cac464199a07a9d3863160a48 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:55.146205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.156136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc1345-37d7-41a9-adba-a05cc7eb63f7 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:55.212502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:55.213121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.213399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.213670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] PATH_INFO: `/v3/users/2736ac24707942448667e4e1913aa8b8` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.214809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2736ac24707942448667e4e1913aa8b8)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:55.216769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:55.262738 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2736ac24707942448667e4e1913aa8b8'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.274414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2736ac24707942448667e4e1913aa8b8'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.280863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2736ac24707942448667e4e1913aa8b8'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.288513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2736ac24707942448667e4e1913aa8b8'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.289170 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-10eca574-11bc-40b9-9ab6-022fef16a74c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2736ac24707942448667e4e1913aa8b8'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.295289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1209/2427] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:01:55 2022] DELETE /identity/v3/users/2736ac24707942448667e4e1913aa8b8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:55.296281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:55.570360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.571829 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:55.585360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b545b5d-8916-4cd1-820d-1383636fcdda admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:55.642022 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:55.642683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.642962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.643230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] PATH_INFO: `/v3/projects/fef953b584de44a994565aed2c8fd5ea` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.644340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef953b584de44a994565aed2c8fd5ea)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:55.646248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:55.691618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef953b584de44a994565aed2c8fd5ea', 'request_id': 'req-0d816c7d-14d3-4447-ba50-5f1087c53520'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.697674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef953b584de44a994565aed2c8fd5ea', 'request_id': 'req-0d816c7d-14d3-4447-ba50-5f1087c53520'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.703783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef953b584de44a994565aed2c8fd5ea', 'request_id': 'req-0d816c7d-14d3-4447-ba50-5f1087c53520'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.716007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] The token cache is being invalidate because project fef953b584de44a994565aed2c8fd5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:01:55.716297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.716917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0d816c7d-14d3-4447-ba50-5f1087c53520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:01:55.718385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1219/2428] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:01:55 2022] DELETE /identity/v3/projects/fef953b584de44a994565aed2c8fd5ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:55.723532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.723833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.724236 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.744552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:55.745811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:55.807285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1210/2429] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:55.815145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.815437 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.815715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.835956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:55.837150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:55.844952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1220/2430] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:55.849619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.859356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb7a89-ed53-42af-8962-61a1c84c1e16 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:55.864167 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:55.865511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.865835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.866110 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.866913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:01:55.867215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:55.868802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:55.880156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1211/2431] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:01:55.884246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.893557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a5fad-c45a-4ffd-8764-9f36f72ebd83 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:55.898309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:55.898990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:55.899271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:55.899542 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:55.900167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:55.902043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:55.918946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:55.934881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1221/2432] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:01:55 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:01:55.939141 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:55.949079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6569e44-3903-42f7-a183-c854cc54d43c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:56.005067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:56.005682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.005969 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.006238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.006884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:56.008804 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:56.029542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1212/2433] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:01:55 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:01:56.033773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:56.044481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64c6c-d4a2-4efd-b15a-de4b23a828e9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:56.049004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:56.049622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.049895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.050163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.050832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:56.052588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:56.062730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1222/2434] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:56.066757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:56.077404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6e19-869c-4b9b-b794-5e75515ba74c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:56.081943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:56.082559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.082836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.083105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.084326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:56.085923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dc055-56fb-4ac2-826f-3c6870e73628 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:56.096428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1213/2435] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:01:56.100636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:56.110712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f462e5a-68f9-472d-a172-8925d5154859 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:56.115534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:56.116199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.116463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.116738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] PATH_INFO: `/v3/projects/ad0f500f01f14ea6965b1e5975fd7171/users/4aeb19790b34457a9217b92bc15c7095/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.132030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0f500f01f14ea6965b1e5975fd7171, user_id=4aeb19790b34457a9217b92bc15c7095, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:56.134703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:56.148134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1223/2436] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:01:56 2022] PUT /identity/v3/projects/ad0f500f01f14ea6965b1e5975fd7171/users/4aeb19790b34457a9217b92bc15c7095/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:01:56.159836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.160354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.160354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.188811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] MFA Rules not processed for user `4aeb19790b34457a9217b92bc15c7095`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:01:56.190021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:56.243713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:01:56.255633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1214/2437] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:01:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:01:56.266211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:01:56.267891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:01:56.324893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91c8aa-af81-4757-83e1-37d31e04cd3f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:01:56.328472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:01:56.329069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:01:56.329349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:01:56.329619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:01:56.338259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:01:56.339702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eae3f9-d0e0-48e0-9650-f8d4ce5a99ac service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:01:56.348391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1224/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:01:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:02:00.402386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:00.463705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc055b3-0bf9-4963-bbd2-b67f38070398 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:00.469600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:00.469600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:00.469600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:00.469600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] PATH_INFO: `/v3/users/4aeb19790b34457a9217b92bc15c7095` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:00.470560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aeb19790b34457a9217b92bc15c7095)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:00.472480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:00.517812 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aeb19790b34457a9217b92bc15c7095'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:00.530235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aeb19790b34457a9217b92bc15c7095'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:00.537342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aeb19790b34457a9217b92bc15c7095'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:00.544148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aeb19790b34457a9217b92bc15c7095'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:00.544771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aeb19790b34457a9217b92bc15c7095'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:00.550461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1215/2439] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:02:00 2022] DELETE /identity/v3/users/4aeb19790b34457a9217b92bc15c7095 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:02:00.555567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:00.556076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:00.556226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:00.575827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:00.577036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:00.636240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1225/2440] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:00.645727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:00.647967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:00.705187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff690bd-0932-4f98-a939-26db830d5b08 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:00.709796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:00.710405 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:00.710739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:00.711015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:00.720053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:00.721486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b177-3ed7-4b39-a8d9-6e76e6976f1a service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:00.731354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1216/2441] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:02:01.010823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.012253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:01.025499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b147b8-6f1f-4a8d-bf3f-51e4d28a42bf None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.030095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.030751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.031031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.031298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] PATH_INFO: `/v3/projects/ad0f500f01f14ea6965b1e5975fd7171` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.032403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0f500f01f14ea6965b1e5975fd7171)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.034297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.083543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0f500f01f14ea6965b1e5975fd7171', 'request_id': 'req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:01.089230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0f500f01f14ea6965b1e5975fd7171', 'request_id': 'req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:01.090302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:01.097149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0f500f01f14ea6965b1e5975fd7171', 'request_id': 'req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:01.110743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] The token cache is being invalidate because project ad0f500f01f14ea6965b1e5975fd7171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:02:01.111037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:01.111670 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-032718b7-a159-4bb3-b6ef-4f5d1e25b2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:02:01.113198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1226/2442] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:02:01 2022] DELETE /identity/v3/projects/ad0f500f01f14ea6965b1e5975fd7171 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:02:01.118760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.119045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.119323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.138693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:01.139902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:01.149566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:01.201374 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1217/2443] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:01.211451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.211750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.212032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.232075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:01.233293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:01.243246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1227/2444] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:01.247582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.257449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894a0b2-a147-4d8b-adf0-fdcc81f88f63 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.261932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.262540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.262830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.263095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.263754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:02:01.264077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.265655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.277227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1218/2445] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:02:01.281384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.291174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced85e36-33ff-4b78-983b-6db33e240a8a None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.295724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.296317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.296593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.296862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.297487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.299430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.328219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1228/2446] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:02:01 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:02:01.332411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.343117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecca7b-3cf5-4763-9489-08c3e7c0a169 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.401100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.401709 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.401989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.402260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.402989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.404946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.425054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1219/2447] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:02:01 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:02:01.429686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.440018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b57a1-d67e-43e6-a267-2e023af6dcb9 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.445515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.446114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.446398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.446712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.447364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.448977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.463410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1229/2448] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:02:01.467343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.482854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a9501-daf1-4f4c-ac09-5b5eb2f568ec admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.487634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.488250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.488533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.488814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.490257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.492552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07254cd-8058-4083-88b4-f9bd05a84292 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.504025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1220/2449] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:02:01.509743 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.519909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a194d6-aadb-4170-a52b-e41af0b54aef admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.525057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.525702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.525978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.526248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] PATH_INFO: `/v3/projects/2862c2760b6548ae95e2ce59470ede1d/users/72ff513b1b924f54818d7d3e38eb4a0b/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.541964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2862c2760b6548ae95e2ce59470ede1d, user_id=72ff513b1b924f54818d7d3e38eb4a0b, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.544635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.558932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1230/2450] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:02:01 2022] PUT /identity/v3/projects/2862c2760b6548ae95e2ce59470ede1d/users/72ff513b1b924f54818d7d3e38eb4a0b/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:02:01.564056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d1646-33c8-49a1-be1b-2a6086963e5a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.564365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d1646-33c8-49a1-be1b-2a6086963e5a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.564718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d1646-33c8-49a1-be1b-2a6086963e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.589118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fe6d1646-33c8-49a1-be1b-2a6086963e5a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:01.590337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fe6d1646-33c8-49a1-be1b-2a6086963e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:01.651899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1221/2451] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:01.661686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:01.663553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:01.728541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5b109-e570-46a9-93cb-563410c2e283 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:01.732092 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:01.732702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:01.732984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:01.733259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:01.744434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:01.745865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:01.760684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1231/2452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:02:06.008583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.009114 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.009255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.039922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] MFA Rules not processed for user `72ff513b1b924f54818d7d3e38eb4a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:06.041072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:06.050907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:06.113971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1222/2453] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:06.118126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.127735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40acce-7c05-4b6b-a592-fcf83d2a994c service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.132938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.133547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.133827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.134106 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.134797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.136882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.165388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1232/2454] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:02:06 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:02:06.169281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.180398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae8c6-ef61-421d-af29-ea9127b194fa None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.242262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.242941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.243367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.243643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.244286 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.246276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.272376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1223/2455] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:02:06 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:02:06.274335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.287520 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c83e1-f2a5-41f2-8d04-bffb3ab8f44a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.292352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.292967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.293246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.293522 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.294356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.297049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.309188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1233/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:02:06.313303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.323688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cc803-56c2-4c27-9938-2432f18be334 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.328403 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.329055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.329337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.329600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] PATH_INFO: `/v3/projects/9c1b90516f3a43329901cb2fa0565c89/users/f21ecb7ab0a64c75bca32a6d1ae58f60/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.346008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1b90516f3a43329901cb2fa0565c89, user_id=f21ecb7ab0a64c75bca32a6d1ae58f60, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.349332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.364063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1224/2457] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:02:06 2022] PUT /identity/v3/projects/9c1b90516f3a43329901cb2fa0565c89/users/f21ecb7ab0a64c75bca32a6d1ae58f60/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:02:06.378879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.391513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b3ba0-1945-4ada-8a3e-649565754450 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.448227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:06.459865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.460616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.460976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.461313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.462249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.463912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b543145-7273-4e92-aa76-67ccdd77c43a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.475366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1234/2458] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:02:06.480005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:06.494127 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeed30-7ddf-45e5-9e45-942416e6d4eb admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:06.498935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:06.499676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:06.500021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:06.500356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] PATH_INFO: `/v3/projects/9c1b90516f3a43329901cb2fa0565c89/users/f21ecb7ab0a64c75bca32a6d1ae58f60/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:06.502317 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1b90516f3a43329901cb2fa0565c89, user_id=f21ecb7ab0a64c75bca32a6d1ae58f60, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:06.505649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:06.517165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1225/2459] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:02:06 2022] PUT /identity/v3/projects/9c1b90516f3a43329901cb2fa0565c89/users/f21ecb7ab0a64c75bca32a6d1ae58f60/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:02:10.851223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3aafa-fe52-47fb-981a-610afc796cb5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:10.851754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3aafa-fe52-47fb-981a-610afc796cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:10.851754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3aafa-fe52-47fb-981a-610afc796cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:10.881555 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bba3aafa-fe52-47fb-981a-610afc796cb5 None None] MFA Rules not processed for user `f21ecb7ab0a64c75bca32a6d1ae58f60`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:10.883006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bba3aafa-fe52-47fb-981a-610afc796cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:10.951866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1235/2460] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:10.971185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:10.974460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:11.051065 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:11.058057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd4ca2-8443-4ac3-af45-1b33764fd5fb admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:11.064344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:11.065098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:11.065489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:11.065837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:11.131017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:11.132677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9a219-7e6b-4b3c-ac6e-6d0d3970972b service nova] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:11.142466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1226/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:02:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:02:12.201550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745f33-9ed8-4b30-9497-df919a5b4db3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:12.202034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745f33-9ed8-4b30-9497-df919a5b4db3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:12.202153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75745f33-9ed8-4b30-9497-df919a5b4db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:12.221251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-75745f33-9ed8-4b30-9497-df919a5b4db3 None None] MFA Rules not processed for user `72ff513b1b924f54818d7d3e38eb4a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:12.222453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-75745f33-9ed8-4b30-9497-df919a5b4db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:12.229366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1236/2462] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:21.137864 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=27 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:22.215877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=42 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:22.927982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023bf62-4971-4030-a994-15376bf1ffce None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:22.928776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023bf62-4971-4030-a994-15376bf1ffce None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:22.928776 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023bf62-4971-4030-a994-15376bf1ffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:22.947466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0023bf62-4971-4030-a994-15376bf1ffce None None] MFA Rules not processed for user `16e5bc69062549e5bebb5b33e1be492e`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:22.948665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0023bf62-4971-4030-a994-15376bf1ffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:23.001937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1227/2463] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:32.995468 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=25 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:02:51.547987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:51.548521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:51.548521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:51.567969 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] MFA Rules not processed for user `f58a59c389cf4b3198a6c452eb2772d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:51.569167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:51.623269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1237/2464] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:02:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:54.469525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:54.470257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:54.470589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:54.509572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:02:54.511139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:54.578385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1228/2465] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 12:02:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:02:54.632871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:02:54.635085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:02:54.725003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbd096-088f-4136-921e-bec50c2b074c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:02:54.730788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:02:54.731400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:02:54.731943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:02:54.731943 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:02:54.741641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:02:54.743297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:02:54.752145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1238/2466] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:02:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:03:00.580216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:00.582007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:00.648188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80787a-d9df-4b29-a912-1f63effebdbf None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:00.652628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:00.653277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:00.653561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:00.653848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:00.655759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:00.671384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:00.672872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:00.690303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1229/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:02.568565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:02.583075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380f9a9-206a-46f4-9a97-3ddf14bc23ff service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:02.587534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:02.588315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:02.588622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:02.588832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:02.591161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:02.610849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:02.613140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:02.637723 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1239/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:03:02.843634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:02.853342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c125e-d3ae-48b3-bc35-39d9dff73da6 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:02.857974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:02.858668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:02.858922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:02.859386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:02.871849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:02.873473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd1bc2-727c-4713-88ba-997d3e3fb82a service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:02.887257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1230/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:03:10.288094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:10.289559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:10.303658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082cf32-c8b7-4bbf-bf4d-751b87eefdce service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:10.309368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:10.309993 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:10.310273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:10.310574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:10.312275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:10.391416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:10.393594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bb7b5-a0e2-41d6-961a-763118b84df2 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:10.399069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:10.414692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1240/2470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:12.641811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.642423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.642423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.643801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1231/2471] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 12:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 12:03:12.650409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a396-8b27-4616-b6ef-039ee641a51c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.650892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a396-8b27-4616-b6ef-039ee641a51c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.650985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5a396-8b27-4616-b6ef-039ee641a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.690225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-04c5a396-8b27-4616-b6ef-039ee641a51c None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:12.691396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-04c5a396-8b27-4616-b6ef-039ee641a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:12.700809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1241/2472] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 12:03:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:03:12.712355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:12.714151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:12.778497 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ae3cf-e80e-44c0-b410-571974ae593b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:12.782036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:12.782675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.782970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.783748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.796912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:12.799957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3ea25-73cd-449f-845c-19af471e976d service glance] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:12.811072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1232/2473] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:12.879552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2946-33e4-4698-93a3-9579437d3bdd None None] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.879946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2946-33e4-4698-93a3-9579437d3bdd None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.880365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2946-33e4-4698-93a3-9579437d3bdd None None] PATH_INFO: `/` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.881976 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1242/2474] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 12:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 12:03:12.886601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e4a-2902-422a-8ec1-1201d01370e2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.886886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e4a-2902-422a-8ec1-1201d01370e2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.887158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76e4a-2902-422a-8ec1-1201d01370e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.906689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cef76e4a-2902-422a-8ec1-1201d01370e2 None None] MFA Rules not processed for user `ca58b9ac9151436aad5a9546fdbe6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:12.907883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cef76e4a-2902-422a-8ec1-1201d01370e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:12.914950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1233/2475] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 12:03:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:03:12.927227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb2946-33e4-4698-93a3-9579437d3bdd None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:12.937939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb2946-33e4-4698-93a3-9579437d3bdd None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:12.941540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:12.942068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:12.942346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:12.942646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:12.949848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:12.951317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93de94-0158-40c8-848a-b47a0083f17d service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:12.959144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1243/2476] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:13.042597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f1b-f2ff-4477-82b8-442293220104 None None] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:13.042882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f1b-f2ff-4477-82b8-442293220104 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:13.043156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255f1b-f2ff-4477-82b8-442293220104 None None] PATH_INFO: `/` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:13.044524 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1234/2477] 10.222.0.15 () {54 vars in 921 bytes} [Mon Sep 12 12:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 12:03:13.049757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f7d5-1bab-4d31-be0d-3982a40cc072 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:13.050107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f7d5-1bab-4d31-be0d-3982a40cc072 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:13.050584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f7d5-1bab-4d31-be0d-3982a40cc072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:13.070831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ff48f7d5-1bab-4d31-be0d-3982a40cc072 None None] MFA Rules not processed for user `7e45de636b0e4f35aa279718cc129923`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:13.072049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ff48f7d5-1bab-4d31-be0d-3982a40cc072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:13.082734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1244/2478] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Sep 12 12:03:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:03:13.093346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255f1b-f2ff-4477-82b8-442293220104 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:13.105455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255f1b-f2ff-4477-82b8-442293220104 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:13.109604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:13.110219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:13.110499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:13.110805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:13.121782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:13.123241 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:13.134337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1235/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:15.023836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:15.024369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:15.024369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:15.044604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:15.046206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:15.057150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1245/2480] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 12:03:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:03:15.070131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:15.087222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1178750-0f26-4fbb-b71e-0cd021ffcfa8 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:15.093218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:15.093829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:15.094106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:15.094375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:15.101391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:15.104710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:15.107399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0dfd80-1dba-4100-9e6e-890545029ed6 service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:15.128828 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1236/2481] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:03:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:03:25.037875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=25 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:25.116798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:32.353650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:32.363462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ea11-d380-4fc1-83a6-ed4f6ea1c8a6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:32.367099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:32.367642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:32.367922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:32.368196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:32.369928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:32.379907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:32.381348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:32.389421 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1246/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:03:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:03:39.828180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d60c0-1b3b-4bdb-b0ff-4befe2df0d6b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:39.828720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d60c0-1b3b-4bdb-b0ff-4befe2df0d6b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:39.828898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d60c0-1b3b-4bdb-b0ff-4befe2df0d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:39.878035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-3f8d60c0-1b3b-4bdb-b0ff-4befe2df0d6b None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:39.879267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-3f8d60c0-1b3b-4bdb-b0ff-4befe2df0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:39.895915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1237/2483] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:03:39.907854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:03:39.929741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890fc0-804b-4e75-a6c2-4e5a540aa939 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:03:39.935019 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:03:39.935624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:39.936008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:39.936176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:39.945263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:03:39.946875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:03:39.954938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1247/2484] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:03:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:03:49.892493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=18 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:49.951712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=16 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:03:51.097958 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:03:51.098493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:03:51.098732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:03:51.118101 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] MFA Rules not processed for user `fb4bb9cd41cd46ddb87a746e732a948f`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:03:51.119394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:03:51.176433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1238/2485] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:03:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:04:01.169181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=25 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:07.534946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:07.548242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892c59-d588-4ab3-b0f6-1b3dd137888d service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:07.568897 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:07.569524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:07.569808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:07.570083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:07.572068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:07.584397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:07.585968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:07.597562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1248/2486] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:04:11.253337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:11.264629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a139-ec40-4c44-b2f5-cb70c27976e7 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:11.269202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:11.269813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:11.270091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:11.270358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:11.272426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:11.334308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:11.335764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:11.345408 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1239/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:04:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:04:17.592778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=6 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:21.341140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=30 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:49.553784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:49.567444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510f08a-b02d-4621-973e-a455c565867c service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:49.573671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:49.574301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:49.574648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:49.574931 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:49.577047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:49.607823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:49.609739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:49.624930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1249/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:04:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:04:54.846035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:54.854534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479faa12-61eb-440c-a908-6ee14eb5fe10 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:54.858787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:54.859377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:54.859654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:54.859922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:54.861630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:54.922698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:54.924115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:54.933810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1240/2489] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:04:57.810560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:57.812254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:57.826551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff50a5-b517-4f5b-86ab-b89a59de51a0 service nova] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:57.831113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:57.831767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:57.832056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:57.832351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] PATH_INFO: `/v3/users/16e5bc69062549e5bebb5b33e1be492e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:57.833610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e5bc69062549e5bebb5b33e1be492e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:57.835888 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:57.888231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e5bc69062549e5bebb5b33e1be492e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:57.901576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e5bc69062549e5bebb5b33e1be492e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:57.915360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e5bc69062549e5bebb5b33e1be492e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:57.916113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e5bc69062549e5bebb5b33e1be492e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:57.923875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-88a2c059-a35b-42d0-8f71-11e86c130bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e5bc69062549e5bebb5b33e1be492e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:57.930258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1250/2490] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:04:57 2022] DELETE /identity/v3/users/16e5bc69062549e5bebb5b33e1be492e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:04:58.110266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.120347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97508b6-6f68-43b7-9d9b-463de97cda93 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.183377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.184574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.184853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.185129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] PATH_INFO: `/v3/projects/6e8a7ef82545435fa36586761e3c85c3` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.186267 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8a7ef82545435fa36586761e3c85c3)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.188513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.233686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8a7ef82545435fa36586761e3c85c3', 'request_id': 'req-4c5a72ec-88d1-402a-8f37-25e898c552c0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:58.240897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8a7ef82545435fa36586761e3c85c3', 'request_id': 'req-4c5a72ec-88d1-402a-8f37-25e898c552c0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:58.247026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8a7ef82545435fa36586761e3c85c3', 'request_id': 'req-4c5a72ec-88d1-402a-8f37-25e898c552c0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:58.258157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] The token cache is being invalidate because project 6e8a7ef82545435fa36586761e3c85c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:04:58.258442 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:58.259107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-4c5a72ec-88d1-402a-8f37-25e898c552c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:04:58.260572 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1241/2491] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:04:58 2022] DELETE /identity/v3/projects/6e8a7ef82545435fa36586761e3c85c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:04:58.265674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.265958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.266237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.287313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:04:58.288524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:58.346880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1251/2492] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:04:58.356460 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.356745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.357019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.378002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:04:58.379428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:58.388466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1242/2493] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:04:58.392606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.404297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c842f49-096f-4ddf-b39b-d25279b46c90 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.409157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.409759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.410039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.410314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.411009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:04:58.411314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.412925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.425531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1252/2494] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:04:58.429620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.439577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19df776-01c7-4bc3-be46-bd135ffa2263 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.444477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.445074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.445356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.445637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.446272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.448243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.463613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:58.487184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1243/2495] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:04:58 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:04:58.491278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.501311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc165-bc8c-44fc-a5dc-03010886232f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.544659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:58.558815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.559509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.559790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.560065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.560730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.562926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.583702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1253/2496] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:04:58 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:04:58.587715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.598223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205c9d-5d7a-4fa4-bd0b-8a4b8cb99254 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.602700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.603293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.603567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.603837 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.604471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.606920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.616642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1244/2497] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:04:58.620453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.630360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb678389-8ef5-4e63-8fb7-43be6e9c86f6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.634989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.635579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.635866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.636139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.636884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.638461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a4762-4d3a-406a-ad08-165731a8fccc admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.649034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1254/2498] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:04:58.653616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.663411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b93647-a9c4-4a84-aabd-0540765619e7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.668520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.669151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.669438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.669690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] PATH_INFO: `/v3/projects/dfcbec8118db4ea5a1cb868faf7f97f3/users/bc3cdc38ec5b4a44bf9c0b6b9e37706f/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.683946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcbec8118db4ea5a1cb868faf7f97f3, user_id=bc3cdc38ec5b4a44bf9c0b6b9e37706f, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.686660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.699425 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1245/2499] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:04:58 2022] PUT /identity/v3/projects/dfcbec8118db4ea5a1cb868faf7f97f3/users/bc3cdc38ec5b4a44bf9c0b6b9e37706f/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:04:58.704427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1311b5-fdc4-48d2-b61c-c42ba889cc90 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.704713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1311b5-fdc4-48d2-b61c-c42ba889cc90 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.704987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1311b5-fdc4-48d2-b61c-c42ba889cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.725119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2d1311b5-fdc4-48d2-b61c-c42ba889cc90 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:04:58.726388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2d1311b5-fdc4-48d2-b61c-c42ba889cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:58.787559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1255/2500] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:04:58.797843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:58.799565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:58.857372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5377-43ae-49e1-a0ef-674b82d93fce admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:58.860954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:58.861555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:58.861832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:58.862102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:58.870997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:58.872423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:58.881690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1246/2501] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:04:59.508163 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.508713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.508713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.537532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] MFA Rules not processed for user `bc3cdc38ec5b4a44bf9c0b6b9e37706f`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:04:59.538838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:04:59.611981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1256/2502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:04:59.616565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.627229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418012-d41a-48eb-90ad-5e4858e0445b service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.631846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.632533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.632886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.634319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.635017 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.636980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.672652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1247/2503] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:04:59 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:04:59.674057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.685706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a858-46c0-4177-9099-9df1ef805855 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.739231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:04:59.748157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.749297 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.749939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.750460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.751467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.753413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.776036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1257/2504] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:04:59 2022] POST /identity/v3/users => generated 309 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:04:59.780044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.790699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01a004-1fb1-4ba6-99d4-331c7ff9b5fe admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.796185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.796925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.797279 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.797628 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.798359 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.800073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.811803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1248/2505] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:04:59.816205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.827286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb4258-c273-4da9-84a0-4c045ada6fc0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.832060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.833237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.833668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.834018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] PATH_INFO: `/v3/projects/43d21afb75744a60a30d067900371900/users/62f4b21a12284984a59c4571a1bed1a0/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.849281 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d21afb75744a60a30d067900371900, user_id=62f4b21a12284984a59c4571a1bed1a0, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.851956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.864729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1258/2506] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:04:59 2022] PUT /identity/v3/projects/43d21afb75744a60a30d067900371900/users/62f4b21a12284984a59c4571a1bed1a0/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:04:59.868742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.879272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc8bb-93a4-43bf-bfb6-cc6700d3103b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.935177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.935782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.936061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.936330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.936967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.938894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9f908-8c69-4131-8bb3-55c63663e59c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.948423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1249/2507] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:04:59.952942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:04:59.963148 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93205-ef8b-46ef-9251-b6914dcc39a7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:04:59.968384 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:04:59.969032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:04:59.969313 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:04:59.969588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] PATH_INFO: `/v3/projects/43d21afb75744a60a30d067900371900/users/62f4b21a12284984a59c4571a1bed1a0/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:04:59.971568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d21afb75744a60a30d067900371900, user_id=62f4b21a12284984a59c4571a1bed1a0, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:04:59.974192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:04:59.985478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1259/2508] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:04:59 2022] PUT /identity/v3/projects/43d21afb75744a60a30d067900371900/users/62f4b21a12284984a59c4571a1bed1a0/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:00.637383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:00.637879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:00.638682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:00.677063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] MFA Rules not processed for user `62f4b21a12284984a59c4571a1bed1a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:00.678551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:00.685910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:00.758541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1250/2509] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:01.224420 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:01.226362 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:01.318802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f187-9b5f-4ffd-b178-f9d56c2a28b7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:01.324260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:01.324902 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:01.325183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:01.325455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:01.327222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:01.339727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:01.341479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:01.351627 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1260/2510] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:01.823105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:01.898269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f2e3b-6469-4071-a95e-e6fb744b3035 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:01.903464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:01.904096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:01.904398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:01.904670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] PATH_INFO: `/v3/users/bc3cdc38ec5b4a44bf9c0b6b9e37706f` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:01.905797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3cdc38ec5b4a44bf9c0b6b9e37706f)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:01.907763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:01.957061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3cdc38ec5b4a44bf9c0b6b9e37706f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:01.969318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3cdc38ec5b4a44bf9c0b6b9e37706f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:01.976368 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3cdc38ec5b4a44bf9c0b6b9e37706f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:01.983426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3cdc38ec5b4a44bf9c0b6b9e37706f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:01.984077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3cdc38ec5b4a44bf9c0b6b9e37706f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:01.989851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1251/2511] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:01 2022] DELETE /identity/v3/users/bc3cdc38ec5b4a44bf9c0b6b9e37706f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:02.168341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:02.180181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a4a9-aca9-4d57-a316-8c23815a6f83 service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:02.239914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:02.240580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.240732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.241009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] PATH_INFO: `/v3/projects/dfcbec8118db4ea5a1cb868faf7f97f3` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.242127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcbec8118db4ea5a1cb868faf7f97f3)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:02.244070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:02.287107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:02.296269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcbec8118db4ea5a1cb868faf7f97f3', 'request_id': 'req-e2dbab66-e7b6-44a2-8d17-cfc45992645f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.302432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcbec8118db4ea5a1cb868faf7f97f3', 'request_id': 'req-e2dbab66-e7b6-44a2-8d17-cfc45992645f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.308605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcbec8118db4ea5a1cb868faf7f97f3', 'request_id': 'req-e2dbab66-e7b6-44a2-8d17-cfc45992645f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.321971 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] The token cache is being invalidate because project dfcbec8118db4ea5a1cb868faf7f97f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:02.322255 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.322909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.325343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1261/2512] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:02 2022] DELETE /identity/v3/projects/dfcbec8118db4ea5a1cb868faf7f97f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:02.329584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:02.331034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:02.344549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132aaa1e-1888-4d42-a116-b921207e8c93 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:02.362953 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:02.405746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:02.406387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.406700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.406974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] PATH_INFO: `/v3/users/62f4b21a12284984a59c4571a1bed1a0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.408222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f4b21a12284984a59c4571a1bed1a0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:02.410162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:02.458552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f4b21a12284984a59c4571a1bed1a0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.471561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f4b21a12284984a59c4571a1bed1a0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.478646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f4b21a12284984a59c4571a1bed1a0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.485989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f4b21a12284984a59c4571a1bed1a0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.486660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2ec1af19-4d2a-4805-badd-c7810dd62bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f4b21a12284984a59c4571a1bed1a0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.492534 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1252/2513] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:02 2022] DELETE /identity/v3/users/62f4b21a12284984a59c4571a1bed1a0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:02.662576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:02.664245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:02.691688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbab66-e7b6-44a2-8d17-cfc45992645f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:02.751631 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:02.752406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.752711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.753118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] PATH_INFO: `/v3/projects/43d21afb75744a60a30d067900371900` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.754248 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d21afb75744a60a30d067900371900)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:02.756194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:02.804657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d21afb75744a60a30d067900371900', 'request_id': 'req-74706b86-be98-4f59-943f-2699e73b2200'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.811236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d21afb75744a60a30d067900371900', 'request_id': 'req-74706b86-be98-4f59-943f-2699e73b2200'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.818068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d21afb75744a60a30d067900371900', 'request_id': 'req-74706b86-be98-4f59-943f-2699e73b2200'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.833398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] The token cache is being invalidate because project 43d21afb75744a60a30d067900371900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:02.833699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.834456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-74706b86-be98-4f59-943f-2699e73b2200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:02.836068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1262/2514] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:02 2022] DELETE /identity/v3/projects/43d21afb75744a60a30d067900371900 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:02.841734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.842270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.842270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.862215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:02.863453 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:02.923260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1253/2515] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:02.931127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.931402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.932347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.958352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:02.959776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:02.970768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1263/2516] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:02.973637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:02.986223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe8856-6ed2-42df-9df1-b1202f887e86 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:02.992052 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:02.992817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:02.993199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:02.993595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:02.994386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:02.994817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:02.996584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.008929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1254/2517] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:03.013397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.022813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f23c4-0284-4add-897f-439bf7cc9cfb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.028102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.028791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.029162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.029510 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.030210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.032220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.061205 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1264/2518] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:03.065821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.075854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7048f8f-9c5b-4177-80ee-3c46f9e62d93 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.109686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:03.144552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.145285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.145569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.145838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.146481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.148449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.173160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1255/2519] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:03.174796 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.185581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273c655-7aa2-43e3-bf56-fbe20102c7a7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.191381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.192115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.192618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.192766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.193468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.195424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.206991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1265/2520] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:03.214582 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.226318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb0c3-1197-4946-b761-b94c5d8e6646 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.231341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.231972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.232254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.232527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.233190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.234878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98a9c1-6778-41d0-91f5-aef137a78849 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.245742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1256/2521] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:03.250962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.262876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcd250-5ab8-427c-acab-3874dbb9206b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.267639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.268342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.268773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.269055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] PATH_INFO: `/v3/projects/1c031be688704fdd866f3f8a451678fe/users/5f20b38f2f364f4fadf71b61f07bbe10/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.287768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c031be688704fdd866f3f8a451678fe, user_id=5f20b38f2f364f4fadf71b61f07bbe10, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.291042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.294551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:03.314589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1266/2522] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:03 2022] PUT /identity/v3/projects/1c031be688704fdd866f3f8a451678fe/users/5f20b38f2f364f4fadf71b61f07bbe10/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:03.322069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.322353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.322734 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.357121 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] MFA Rules not processed for user `5f20b38f2f364f4fadf71b61f07bbe10`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:03.358360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:03.427043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1257/2523] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:03.431630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.498064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b4d1-8f28-44a8-98cb-b9775689c726 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.502937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.503552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.503826 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.504093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.504734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.506651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.534327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1267/2524] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:03.538318 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.550463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4c991-cb40-4ca7-81ae-eaf8ec179d01 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.611182 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.611798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.612079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.612353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.612992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.614963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.640604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1258/2525] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:03.641802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.654738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad23af-f997-4c25-b06d-d9faae283661 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.659471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.660078 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.660360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.660639 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.661289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.662924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.674144 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1268/2526] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:03.678138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.689457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8092c-8a70-4590-a3e3-6aecc37d7bf0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.694136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.694939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.695313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.695662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] PATH_INFO: `/v3/projects/ac05c8db55ec4af680f377e1c97ceb39/users/dbafd0da5c064cc2a977ef9fe9fb9b75/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.712416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05c8db55ec4af680f377e1c97ceb39, user_id=dbafd0da5c064cc2a977ef9fe9fb9b75, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.715727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.729309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1259/2527] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:03 2022] PUT /identity/v3/projects/ac05c8db55ec4af680f377e1c97ceb39/users/dbafd0da5c064cc2a977ef9fe9fb9b75/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:03.733968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.744290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13cbc2-93ba-4a06-a3b6-81d61c49cb92 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.811810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.814351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.814697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.815035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.816424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.818145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17d3d3-9c00-45f9-ae73-33fb604441c9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.830747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1269/2528] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:03.835771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:03.841581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:03.850859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d743-6926-4941-89f4-f0caf5e44ea7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:03.857420 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:03.858144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.858424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.858728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] PATH_INFO: `/v3/projects/ac05c8db55ec4af680f377e1c97ceb39/users/dbafd0da5c064cc2a977ef9fe9fb9b75/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.860655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05c8db55ec4af680f377e1c97ceb39, user_id=dbafd0da5c064cc2a977ef9fe9fb9b75, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:03.863378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:03.876380 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1260/2529] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:03 2022] PUT /identity/v3/projects/ac05c8db55ec4af680f377e1c97ceb39/users/dbafd0da5c064cc2a977ef9fe9fb9b75/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:03.886784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:03.887892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:03.888491 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:03.928880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] MFA Rules not processed for user `dbafd0da5c064cc2a977ef9fe9fb9b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:03.930608 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:04.012959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:04.058321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1270/2530] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:04.070591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:04.072432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:04.164372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44832786-c413-4c6d-a9b3-4b18ae9a720e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:04.169676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:04.170811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:04.171235 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:04.171574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:04.189619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:04.192394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423fa1-1893-4938-b899-0b01ecc3d7f0 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:04.208199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1261/2531] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:05.791594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:05.877763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1caaf-32f5-4c50-bda8-88116c5235a6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:05.884360 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:05.885349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:05.885828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:05.886448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] PATH_INFO: `/v3/users/5f20b38f2f364f4fadf71b61f07bbe10` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:05.889280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f20b38f2f364f4fadf71b61f07bbe10)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:05.891930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:05.951531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f20b38f2f364f4fadf71b61f07bbe10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:05.961739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f20b38f2f364f4fadf71b61f07bbe10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:05.976308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f20b38f2f364f4fadf71b61f07bbe10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:05.977155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f20b38f2f364f4fadf71b61f07bbe10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:05.984325 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f20b38f2f364f4fadf71b61f07bbe10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:05.990230 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1271/2532] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:05 2022] DELETE /identity/v3/users/5f20b38f2f364f4fadf71b61f07bbe10 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:05.994715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:05.995191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:05.995393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:06.018457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:06.020363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:06.091022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1262/2533] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:06.103112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:06.104531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:06.170490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df0ffc-3dc0-473f-b34c-2d5a90c0cbda admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:06.174722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:06.175309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:06.175588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:06.175864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:06.187141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:06.188862 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:06.198944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1272/2534] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:06.492377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:06.494096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:06.508540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02969d17-2c19-403f-8185-df77e7ed33c4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:06.514371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:06.515161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:06.515589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:06.515986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] PATH_INFO: `/v3/projects/1c031be688704fdd866f3f8a451678fe` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:06.517277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c031be688704fdd866f3f8a451678fe)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:06.519346 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:06.571937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c031be688704fdd866f3f8a451678fe', 'request_id': 'req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.582999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c031be688704fdd866f3f8a451678fe', 'request_id': 'req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.590010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c031be688704fdd866f3f8a451678fe', 'request_id': 'req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.602934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] The token cache is being invalidate because project 1c031be688704fdd866f3f8a451678fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:06.604384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.605035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.607100 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1263/2535] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:06 2022] DELETE /identity/v3/projects/1c031be688704fdd866f3f8a451678fe => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:06.611530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:06.621406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948107-aa48-4b55-ba8c-2a94867a5623 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:06.630503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:06.689299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:06.689930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:06.690290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:06.690591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] PATH_INFO: `/v3/users/dbafd0da5c064cc2a977ef9fe9fb9b75` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:06.691776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbafd0da5c064cc2a977ef9fe9fb9b75)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:06.693788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:06.750467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbafd0da5c064cc2a977ef9fe9fb9b75'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.758503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbafd0da5c064cc2a977ef9fe9fb9b75'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.774182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbafd0da5c064cc2a977ef9fe9fb9b75'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.774982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbafd0da5c064cc2a977ef9fe9fb9b75'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.782831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a1965a5a-e375-46a2-b499-b2049793cffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbafd0da5c064cc2a977ef9fe9fb9b75'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:06.789194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1273/2536] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:06 2022] DELETE /identity/v3/users/dbafd0da5c064cc2a977ef9fe9fb9b75 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:07.080912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.082484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:07.097249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3b2f4-a35d-4d1b-8b13-adaf7389c443 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.114760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:07.169951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.170492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.170799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.171073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] PATH_INFO: `/v3/projects/ac05c8db55ec4af680f377e1c97ceb39` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.172288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac05c8db55ec4af680f377e1c97ceb39)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.174187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.223930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac05c8db55ec4af680f377e1c97ceb39', 'request_id': 'req-c689985f-921f-4601-bdd7-d0c7af176031'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:07.234710 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac05c8db55ec4af680f377e1c97ceb39', 'request_id': 'req-c689985f-921f-4601-bdd7-d0c7af176031'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:07.240684 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac05c8db55ec4af680f377e1c97ceb39', 'request_id': 'req-c689985f-921f-4601-bdd7-d0c7af176031'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:07.255342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] The token cache is being invalidate because project ac05c8db55ec4af680f377e1c97ceb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:07.255636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:07.256296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-c689985f-921f-4601-bdd7-d0c7af176031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:07.258103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1264/2537] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:07 2022] DELETE /identity/v3/projects/ac05c8db55ec4af680f377e1c97ceb39 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:07.264269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.264750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.264870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.285797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:07.287033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:07.359473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1274/2538] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:07.362936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.362936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.363179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.383680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:07.385198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:07.394394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1265/2539] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:07.401342 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.410107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a9bdf-61de-47fb-addf-323a1339ce1e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.414932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.415636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.415996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.416339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.417051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:07.417424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.419545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.431560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1275/2540] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:07.437008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.447565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81aedf-d3e3-4ad0-ba21-0cc769de9f02 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.453360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.454719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.455021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.455297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.455942 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.458147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.491256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1266/2541] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:07.492683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.503832 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2f9e-72ef-44fb-a1c7-3126aeb9bab6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.517069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:07.565722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.566333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.566648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.566922 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.567554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.569472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.590198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1276/2542] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:07 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:07.595402 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.605771 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f284-5fdc-43de-96bb-ed0590206622 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.611142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.611740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.612018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.612292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.612937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.614560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.625252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1267/2543] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:07.629400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.639351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980907b-9d3e-4885-ac7f-e421f9c3b8ec admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.643925 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.644514 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.644792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.645064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.645705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.647329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b265818-1cbd-497a-bd6c-a89cdea94b68 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.658407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1277/2544] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:07.662598 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.673218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6738-1a5f-48c0-b0fc-73a8ef116a8c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.678010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.678629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.678907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.679178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] PATH_INFO: `/v3/projects/180f271dc4d04009a2ddd31a982b429a/users/5f08cf0be8ae42d79d5e5a33f81620c6/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.693692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] RBAC: Authorizing `identity:create_grant(project_id=180f271dc4d04009a2ddd31a982b429a, user_id=5f08cf0be8ae42d79d5e5a33f81620c6, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.696370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.708637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1268/2545] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:07 2022] PUT /identity/v3/projects/180f271dc4d04009a2ddd31a982b429a/users/5f08cf0be8ae42d79d5e5a33f81620c6/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:07.720326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.720613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.720891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.748951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] MFA Rules not processed for user `5f08cf0be8ae42d79d5e5a33f81620c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:07.750157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:07.813634 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1278/2546] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:07.825753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:07.827447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:07.879614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:07.916254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfb775-9a63-4f52-a4f5-9eeb683912df admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:07.920404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837005-a505-415f-b18f-32932b1577ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:07.921010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837005-a505-415f-b18f-32932b1577ce service glance] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:07.921373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837005-a505-415f-b18f-32932b1577ce service glance] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:07.921654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07837005-a505-415f-b18f-32932b1577ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:07.930707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837005-a505-415f-b18f-32932b1577ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:07.932074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07837005-a505-415f-b18f-32932b1577ce service glance] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:07.941252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1269/2547] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:08.621701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:08.623455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:08.680193 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120b7f-7b8b-4200-afdf-e8a329c55d83 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:08.683766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:08.684371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:08.685196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:08.685472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:08.687208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:08.729557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:08.754715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:08.756160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:08.766688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1279/2548] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:11.829608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837005-a505-415f-b18f-32932b1577ce service glance] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:11.831341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-07837005-a505-415f-b18f-32932b1577ce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:11.850742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07837005-a505-415f-b18f-32932b1577ce service glance] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:11.857437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:11.858180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:11.858419 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:11.858800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] PATH_INFO: `/v3/users/6e3c1f7f6f8649a991d5d8a15eb8826b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:11.859901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3c1f7f6f8649a991d5d8a15eb8826b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:11.861866 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:11.916342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3c1f7f6f8649a991d5d8a15eb8826b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:11.930038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3c1f7f6f8649a991d5d8a15eb8826b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:11.937522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3c1f7f6f8649a991d5d8a15eb8826b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:11.945118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3c1f7f6f8649a991d5d8a15eb8826b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:11.945742 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6f3ea406-e710-4293-8c71-9e8c5516a521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3c1f7f6f8649a991d5d8a15eb8826b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:11.952477 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1270/2549] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:11 2022] DELETE /identity/v3/users/6e3c1f7f6f8649a991d5d8a15eb8826b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:12.131897 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.149090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d200ee5-fa9c-4a32-8b5d-93a27afef51f service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.212920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.213468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.213774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.214046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] PATH_INFO: `/v3/projects/0356e6690066437a86fc69833ff4cb69` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.215178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0356e6690066437a86fc69833ff4cb69)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.217347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.265657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0356e6690066437a86fc69833ff4cb69', 'request_id': 'req-9ba84a04-afb1-453c-80c9-a2fb8635bd70'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:12.271412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0356e6690066437a86fc69833ff4cb69', 'request_id': 'req-9ba84a04-afb1-453c-80c9-a2fb8635bd70'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:12.278220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0356e6690066437a86fc69833ff4cb69', 'request_id': 'req-9ba84a04-afb1-453c-80c9-a2fb8635bd70'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:12.292919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] The token cache is being invalidate because project 0356e6690066437a86fc69833ff4cb69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:12.293266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:12.294875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ba84a04-afb1-453c-80c9-a2fb8635bd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:12.296841 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1280/2550] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:12 2022] DELETE /identity/v3/projects/0356e6690066437a86fc69833ff4cb69 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:12.301839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.302130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.302428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.323231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:12.324437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:12.384163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1271/2551] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:12.392948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.393235 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.393511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.417777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:12.419047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:12.427375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1281/2552] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:12.431661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.442140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1343b3e-d6c9-433b-958a-7f0864bb1f98 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.447062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.447676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.447956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.448252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.448899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:12.449202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.451349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.463390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1272/2553] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:12.467933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.477165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1d6d9-2dda-418b-b461-c542fb0e02bb None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.482162 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.482871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.483149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.483416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.484047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.485924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.515197 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1282/2554] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:12 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:12.519608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.529957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafef2c-3efd-43da-9289-cea3a1409725 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.554449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:12.598894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.600867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.601298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.601622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.602386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.604335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.633253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1273/2555] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:12 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:12.637454 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.648199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857054-c4e0-427d-8fab-20366d440eb5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.652748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.653335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.654252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.654624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.655277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.656887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.667534 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1283/2556] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:12.672421 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.682450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6b07-65f0-4660-9cdc-95c7233d4c0f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.687324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.687920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.688202 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.688475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.689480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.691183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc755-e2fb-43ec-8934-208037895338 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.702575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1274/2557] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:12.706849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.717933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ceb9b-1199-4a91-8b88-62948a2415bd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.722638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.723286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.723564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.723837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] PATH_INFO: `/v3/projects/0d268e52a2f44076aa085841de8bc70d/users/9bc7f6a21901446c8063116cfaf6f1fb/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.738782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d268e52a2f44076aa085841de8bc70d, user_id=9bc7f6a21901446c8063116cfaf6f1fb, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.741424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.754932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1284/2558] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:12 2022] PUT /identity/v3/projects/0d268e52a2f44076aa085841de8bc70d/users/9bc7f6a21901446c8063116cfaf6f1fb/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:12.765868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.766222 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.766490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.793941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] MFA Rules not processed for user `9bc7f6a21901446c8063116cfaf6f1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:12.795946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:12.864391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1275/2559] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:12.875817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:12.877570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:12.882881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:12.959038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77153795-b925-4a56-a1fd-17df2c0f6508 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:12.965177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:12.965797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:12.966123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:12.966502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:12.974801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:12.976320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3864fb-5c93-4151-90fc-362c9c49e155 service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:12.985472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1285/2560] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:13.198749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:13.264068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca439fe-d7bb-4eb1-94f9-ee891b96f785 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:13.268466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:13.269095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:13.269378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:13.269650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] PATH_INFO: `/v3/users/9bc7f6a21901446c8063116cfaf6f1fb` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:13.270816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc7f6a21901446c8063116cfaf6f1fb)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:13.272766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:13.312388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:13.323704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc7f6a21901446c8063116cfaf6f1fb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.338054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc7f6a21901446c8063116cfaf6f1fb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.348441 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc7f6a21901446c8063116cfaf6f1fb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.359116 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc7f6a21901446c8063116cfaf6f1fb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.359835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc7f6a21901446c8063116cfaf6f1fb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.365910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1276/2561] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:13 2022] DELETE /identity/v3/users/9bc7f6a21901446c8063116cfaf6f1fb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:13.370568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:13.371070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:13.371287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:13.391279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:13.392486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:13.462280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1286/2562] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:13.472062 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:13.473617 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:13.537103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb13c48-afaf-4e58-a173-088338e28751 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:13.540869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:13.541504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:13.541779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:13.542045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:13.551193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:13.552654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7621430-0ae2-4603-a925-e94507ea55df service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:13.563158 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1277/2563] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:13.861330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:13.863870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:13.878817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e968147-f95c-4b9f-89b3-2599738b1198 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:13.884244 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:13.884892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:13.885172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:13.885452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] PATH_INFO: `/v3/projects/0d268e52a2f44076aa085841de8bc70d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:13.886633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d268e52a2f44076aa085841de8bc70d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:13.889374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:13.944506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d268e52a2f44076aa085841de8bc70d', 'request_id': 'req-01b17dd2-7da7-430a-a149-28540a3edd5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.949935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d268e52a2f44076aa085841de8bc70d', 'request_id': 'req-01b17dd2-7da7-430a-a149-28540a3edd5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.956426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d268e52a2f44076aa085841de8bc70d', 'request_id': 'req-01b17dd2-7da7-430a-a149-28540a3edd5b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.969026 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] The token cache is being invalidate because project 0d268e52a2f44076aa085841de8bc70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:13.969346 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.969996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-01b17dd2-7da7-430a-a149-28540a3edd5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:13.971606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1287/2564] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:13 2022] DELETE /identity/v3/projects/0d268e52a2f44076aa085841de8bc70d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:13.976706 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:13.977274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:13.977274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:13.996907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:13.998737 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:14.057203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1278/2565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:14.067648 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.067937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.068215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.088990 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:14.090188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:14.098402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1288/2566] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:14.103080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.113939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de9d1-a442-44f5-bef6-33a2657f313f None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.118809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.119449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.119730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.120004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.120655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:14.120959 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.123385 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.135583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1279/2567] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:14.140898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.147666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:14.152102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4e122-706a-4357-a42f-b7c341a67217 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.159151 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.159933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.160378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.160693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.161465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.163414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.191938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1289/2568] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:14.196142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.208633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da87cb-1235-410b-be77-361bad27f081 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.264363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:14.282912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.283581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.283897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.284197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.284841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.286889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.311967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1280/2569] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:14.313010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.323402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a347-d4c8-4e3c-8829-db21862a68da admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.329363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.330007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.330302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.330607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.331462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.332992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.345918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1290/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:14.349570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.359838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf00f1f-79ae-4877-8a11-f5b26be895bc admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.364663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.365276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.365560 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.365835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.366486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.368180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37e38f-4fa3-4113-b5cd-c6457f972866 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.384417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1281/2571] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:14.385610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.396409 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ebb62-8388-4313-8f19-e91c53c19348 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.401278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.402003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.402353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.402668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] PATH_INFO: `/v3/projects/4aaa42db55a9485a89aded36ea2c850b/users/2cba1129c1c74ed4969c3901a6e9c3cd/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.420165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aaa42db55a9485a89aded36ea2c850b, user_id=2cba1129c1c74ed4969c3901a6e9c3cd, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.422885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.436527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1291/2572] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:14 2022] PUT /identity/v3/projects/4aaa42db55a9485a89aded36ea2c850b/users/2cba1129c1c74ed4969c3901a6e9c3cd/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:14.447012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.447296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.447566 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.479157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] MFA Rules not processed for user `2cba1129c1c74ed4969c3901a6e9c3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:14.480796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:14.553796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1282/2573] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:14.558679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.757054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be4250-2956-49a3-ab3d-f3261f2e87a4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.761964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.762635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.762914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.763216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.764096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.766157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.799852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1292/2574] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:14.800987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.811951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028ebe6-37ba-45d0-b9f8-5047f25dd78b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.875908 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.876532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.876815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.877085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.877856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.879852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.902906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1283/2575] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:14 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:14.907887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.918227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe42d4-f684-4921-ba86-1a68ed1770b5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.928096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.928716 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.929010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.929264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.929915 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.931564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:14.943523 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1293/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:14.947878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:14.958067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335575e-cc3b-40aa-be06-a8b485ba9048 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:14.962772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:14.963416 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:14.963691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:14.963964 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] PATH_INFO: `/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748/users/ee5da52d4bd443f38c5e9c5b43db3d7a/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:14.984327 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bea4ccd1154bdd8a1ac1692e28d748, user_id=ee5da52d4bd443f38c5e9c5b43db3d7a, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:14.987082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:15.002995 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1284/2577] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:14 2022] PUT /identity/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748/users/ee5da52d4bd443f38c5e9c5b43db3d7a/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:15.006763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:15.022474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e657d-d714-4638-93e1-6a1d3188ae2f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:15.086047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:15.086047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:15.086047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:15.086047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:15.086581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:15.088290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4589a4d-3c55-4b21-8b5e-3e4dc548f8a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:15.101493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1294/2578] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:15.105720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:15.116721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f63a29-e917-4fe1-972e-218684619d6a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:15.121504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:15.122165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:15.123043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:15.123595 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] PATH_INFO: `/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748/users/ee5da52d4bd443f38c5e9c5b43db3d7a/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:15.125674 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bea4ccd1154bdd8a1ac1692e28d748, user_id=ee5da52d4bd443f38c5e9c5b43db3d7a, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:15.128448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:15.140388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1285/2579] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:15 2022] PUT /identity/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748/users/ee5da52d4bd443f38c5e9c5b43db3d7a/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:15.150789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:15.151161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:15.151498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:15.180299 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] MFA Rules not processed for user `ee5da52d4bd443f38c5e9c5b43db3d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:15.181503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:15.187352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:15.256859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1295/2580] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:15.268312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:15.270020 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:15.275339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:15.334366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0ce4-0479-437d-a8ce-ed394881b08d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:15.338106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:15.338770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:15.339045 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:15.339312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:15.349493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:15.350978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:15.360539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1286/2581] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:15.436848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:15.446700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af145f2-f2a1-4c9d-8336-54a7ffdd11ef None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:15.450202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:15.451463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:15.451750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:15.452053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:15.511674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:15.513028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:15.522250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1296/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:25.356048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=25 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:25.518126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=54 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:29.138849 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:29.148214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b29798-80f0-4924-9535-dba00e80e1d0 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:29.151860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:29.152445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:29.152723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:29.152991 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:29.154692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:29.234256 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:29.235909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57bb02-49e7-447e-ac6a-debf315a13a0 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:29.254434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1287/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:29.686496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:29.747691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4ee74-350d-4869-9f95-262eccc14faa service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:29.752391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:29.753016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:29.753296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:29.753566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] PATH_INFO: `/v3/users/2cba1129c1c74ed4969c3901a6e9c3cd` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:29.754718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cba1129c1c74ed4969c3901a6e9c3cd)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:29.756621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:29.801875 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cba1129c1c74ed4969c3901a6e9c3cd'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:29.809844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cba1129c1c74ed4969c3901a6e9c3cd'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:29.822186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cba1129c1c74ed4969c3901a6e9c3cd'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:29.822835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cba1129c1c74ed4969c3901a6e9c3cd'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:29.830187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cba1129c1c74ed4969c3901a6e9c3cd'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:29.836364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1297/2584] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:29 2022] DELETE /identity/v3/users/2cba1129c1c74ed4969c3901a6e9c3cd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:29.840841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:29.841201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:29.841515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:29.860635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:29.861822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:29.919980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1288/2585] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:29.929740 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:29.931275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:29.989116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccbda4-00cc-41be-8543-b1bae4d3acee admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:29.992945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:29.993693 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:29.993960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:29.994289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:30.005394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:30.006916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:30.017465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1298/2586] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:30.288676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:30.290140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:30.304341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de73da-5846-4a95-8ba6-a3acb4e334ac None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:30.309051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:30.309696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:30.309984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:30.310262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] PATH_INFO: `/v3/projects/4aaa42db55a9485a89aded36ea2c850b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:30.311427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aaa42db55a9485a89aded36ea2c850b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:30.314399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:30.362745 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aaa42db55a9485a89aded36ea2c850b', 'request_id': 'req-127b858f-4bbc-43f4-8b3a-73c328720b13'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.370263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aaa42db55a9485a89aded36ea2c850b', 'request_id': 'req-127b858f-4bbc-43f4-8b3a-73c328720b13'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.376515 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aaa42db55a9485a89aded36ea2c850b', 'request_id': 'req-127b858f-4bbc-43f4-8b3a-73c328720b13'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.377590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:30.389722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] The token cache is being invalidate because project 4aaa42db55a9485a89aded36ea2c850b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:30.390008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.390892 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.392361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1289/2587] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:30 2022] DELETE /identity/v3/projects/4aaa42db55a9485a89aded36ea2c850b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:30.396412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:30.405951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082a48-c9e1-4a16-acf1-7fd9f1353138 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:30.424311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:30.464715 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:30.465280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:30.465537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:30.465807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] PATH_INFO: `/v3/users/ee5da52d4bd443f38c5e9c5b43db3d7a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:30.466950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5da52d4bd443f38c5e9c5b43db3d7a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:30.468955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:30.513008 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5da52d4bd443f38c5e9c5b43db3d7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.520948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5da52d4bd443f38c5e9c5b43db3d7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.536621 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5da52d4bd443f38c5e9c5b43db3d7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.537353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5da52d4bd443f38c5e9c5b43db3d7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.545741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5da52d4bd443f38c5e9c5b43db3d7a'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.552479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1299/2588] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:30 2022] DELETE /identity/v3/users/ee5da52d4bd443f38c5e9c5b43db3d7a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:30.638001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:30.639432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:30.701552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b858f-4bbc-43f4-8b3a-73c328720b13 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:30.753910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:30.754550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:30.754836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:30.755112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:30.756869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:30.822397 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:30.824245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2529223-19e6-4081-85c4-c2de18a6d854 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:30.834762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1290/2589] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:30.883386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:30.884821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:30.907331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f23a2-e6b5-43ea-9971-24ddfec3c5a1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:30.912817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:30.913647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:30.913920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:30.914350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] PATH_INFO: `/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:30.915960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bea4ccd1154bdd8a1ac1692e28d748)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:30.917907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:30.967354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bea4ccd1154bdd8a1ac1692e28d748', 'request_id': 'req-aa27c5a7-74db-4402-8e1d-d096214f30a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.973138 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bea4ccd1154bdd8a1ac1692e28d748', 'request_id': 'req-aa27c5a7-74db-4402-8e1d-d096214f30a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.979463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bea4ccd1154bdd8a1ac1692e28d748', 'request_id': 'req-aa27c5a7-74db-4402-8e1d-d096214f30a7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.993265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] The token cache is being invalidate because project a7bea4ccd1154bdd8a1ac1692e28d748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:30.993561 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.994190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-aa27c5a7-74db-4402-8e1d-d096214f30a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:30.995687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1300/2590] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:30 2022] DELETE /identity/v3/projects/a7bea4ccd1154bdd8a1ac1692e28d748 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:31.001466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.001843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.002183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.022994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:31.024240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:31.076755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:31.086767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1291/2591] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:31.094324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.094791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.095082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.115732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:31.116966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:31.124795 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1301/2592] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:31.129815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.139573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317ca89-516b-4bcf-afcd-82875b476c2a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.144398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.144999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.145280 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.145550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.146194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:31.146493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.148123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.159886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1292/2593] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:31.164220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.174337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1501478-deaa-487b-88fd-50a6b6fe6472 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.179039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.179645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.179924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.180196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.180837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.182775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.200070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:31.220166 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1302/2594] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:31.221478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.235007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276986-eb81-476f-acfc-eb8fe0abee10 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.295128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.295873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.296257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.296621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.298111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.300088 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.326601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1293/2595] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:31.327806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.338217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17bd08-a650-4906-9fde-ed3da5f194d8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.343122 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.343744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.344046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.344347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.345189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.346773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.356789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1303/2596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:31.361678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.372590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18678666-bf37-4b43-b9a4-0e54cb02b600 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.377852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.378532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.378868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.379152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.379808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.381417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe18ede-051c-472f-9ccf-666862996665 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.392731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1294/2597] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:31.397099 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.408274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a37474-49a8-4670-85e6-b27c0188e085 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.413108 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.413755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.414030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.414300 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] PATH_INFO: `/v3/projects/1ac818008ec84feda434efe87aa47236/users/9658246cc41a4b959e59bf8a536ffe79/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.429828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac818008ec84feda434efe87aa47236, user_id=9658246cc41a4b959e59bf8a536ffe79, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.432480 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.445898 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1304/2598] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:31 2022] PUT /identity/v3/projects/1ac818008ec84feda434efe87aa47236/users/9658246cc41a4b959e59bf8a536ffe79/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:31.456251 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.456626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.456992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.487049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] MFA Rules not processed for user `9658246cc41a4b959e59bf8a536ffe79`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:31.488223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:31.555187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1295/2599] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:31.559661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.628231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcab24-5759-489d-a841-1833db1c1ffe admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.639159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.639159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.639159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.639159 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.639569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.639569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.666279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1305/2600] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:31.670463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.682067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e941-eb43-49e0-ab6d-eb7f5e75de89 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.733437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:31.746890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.747671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.748206 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.748645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.749394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.752680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.786284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1296/2601] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:31 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:31.790340 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.804251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588032f2-46dc-4f80-991c-02e7eaba2470 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.810176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.810968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.811528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.811711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.813858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.814504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.826552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1306/2602] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:31.831102 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.856594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796c39f-8718-45bd-8370-8b064bd14faf admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.862507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.863227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.863509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.863786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] PATH_INFO: `/v3/projects/d733f5d350a84d0da303204b4067c78e/users/c54aaca783e94f55bc1b305e5cdce678/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.885725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733f5d350a84d0da303204b4067c78e, user_id=c54aaca783e94f55bc1b305e5cdce678, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.888475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.903806 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1297/2603] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:31 2022] PUT /identity/v3/projects/d733f5d350a84d0da303204b4067c78e/users/c54aaca783e94f55bc1b305e5cdce678/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:31.907754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:31.917967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa3a0c-cbec-495a-a536-9f3dcfb845b2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:31.978866 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:31.979472 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:31.979749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:31.980018 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:31.980654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:31.982251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc2092-247e-4a0b-a03a-e3e5efb78a05 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:31.992804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1307/2604] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:31.996802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:32.007297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316d13-eba9-4d41-b575-987b01ea56fb admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:32.011910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:32.012555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.012831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.013097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] PATH_INFO: `/v3/projects/d733f5d350a84d0da303204b4067c78e/users/c54aaca783e94f55bc1b305e5cdce678/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.015019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d733f5d350a84d0da303204b4067c78e, user_id=c54aaca783e94f55bc1b305e5cdce678, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:32.018178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:32.030343 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1298/2605] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:31 2022] PUT /identity/v3/projects/d733f5d350a84d0da303204b4067c78e/users/c54aaca783e94f55bc1b305e5cdce678/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:32.042059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.042487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.042792 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.071473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] MFA Rules not processed for user `c54aaca783e94f55bc1b305e5cdce678`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:32.072718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:32.095059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:32.152337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1308/2606] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:32.163916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:32.165717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:32.225882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e997b9f-1e5e-4787-a59f-268104d9f77b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:32.229529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:32.230152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.230431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.230759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.241650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:32.244381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64d6c4-05b0-4d5f-aad0-5c061f1a03eb service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:32.253587 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1299/2607] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:32.326055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:32.403586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44aa2e-f198-4059-a422-bff984cab5e8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:32.408879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:32.409513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.409787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.410055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] PATH_INFO: `/v3/users/9658246cc41a4b959e59bf8a536ffe79` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.411269 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9658246cc41a4b959e59bf8a536ffe79)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:32.413185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:32.470968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9658246cc41a4b959e59bf8a536ffe79'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:32.479357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9658246cc41a4b959e59bf8a536ffe79'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:32.494778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9658246cc41a4b959e59bf8a536ffe79'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:32.495429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9658246cc41a4b959e59bf8a536ffe79'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:32.504720 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9658246cc41a4b959e59bf8a536ffe79'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:32.512021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1309/2608] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:32 2022] DELETE /identity/v3/users/9658246cc41a4b959e59bf8a536ffe79 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:32.517223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.517747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.518253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.539596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:32.541006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:32.610567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1300/2609] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:32.620800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:32.622595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:32.701671 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:32.725180 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e170558-5ed8-4cfe-8e8b-ee338ae22c4f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:32.733910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:32.734315 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:32.734868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:32.735033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:32.748082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:32.749529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:32.759835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1310/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:33.076002 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:33.078753 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:33.100844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06169cd2-2565-4055-9cc9-4d2774cc9a66 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:33.114243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:33.114932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:33.115217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:33.115498 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] PATH_INFO: `/v3/projects/1ac818008ec84feda434efe87aa47236` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:33.117842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac818008ec84feda434efe87aa47236)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:33.119819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:33.165131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:33.168834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac818008ec84feda434efe87aa47236', 'request_id': 'req-dde759fc-156e-427a-8ed1-0e93ca268297'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.177573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac818008ec84feda434efe87aa47236', 'request_id': 'req-dde759fc-156e-427a-8ed1-0e93ca268297'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.184333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac818008ec84feda434efe87aa47236', 'request_id': 'req-dde759fc-156e-427a-8ed1-0e93ca268297'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.195648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] The token cache is being invalidate because project 1ac818008ec84feda434efe87aa47236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:33.195935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.196567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.198073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1301/2611] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:33 2022] DELETE /identity/v3/projects/1ac818008ec84feda434efe87aa47236 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:33.201992 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:33.212455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd59af-9218-4a3b-80c9-056c816b396b service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:33.271067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:33.271605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:33.271885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:33.272189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] PATH_INFO: `/v3/users/c54aaca783e94f55bc1b305e5cdce678` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:33.274128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54aaca783e94f55bc1b305e5cdce678)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:33.279113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:33.356906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54aaca783e94f55bc1b305e5cdce678'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.368933 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54aaca783e94f55bc1b305e5cdce678'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.393557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54aaca783e94f55bc1b305e5cdce678'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.394625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54aaca783e94f55bc1b305e5cdce678'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.407433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54aaca783e94f55bc1b305e5cdce678'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:33.416343 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1311/2612] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:33 2022] DELETE /identity/v3/users/c54aaca783e94f55bc1b305e5cdce678 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:33.910685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:33.912144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:33.925939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde759fc-156e-427a-8ed1-0e93ca268297 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.007004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.007660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.007940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.008213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] PATH_INFO: `/v3/projects/d733f5d350a84d0da303204b4067c78e` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.009494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d733f5d350a84d0da303204b4067c78e)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.011934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.067220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.069540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.071885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd733f5d350a84d0da303204b4067c78e', 'request_id': 'req-56cac9e6-3e1a-472b-b33c-a990cd680a51'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.088043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd733f5d350a84d0da303204b4067c78e', 'request_id': 'req-56cac9e6-3e1a-472b-b33c-a990cd680a51'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.096422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea483a6-34f2-457a-b7b1-ed335252357f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.098315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd733f5d350a84d0da303204b4067c78e', 'request_id': 'req-56cac9e6-3e1a-472b-b33c-a990cd680a51'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.110670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] The token cache is being invalidate because project d733f5d350a84d0da303204b4067c78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:34.111079 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.111856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-56cac9e6-3e1a-472b-b33c-a990cd680a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.113513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1302/2613] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:33 2022] DELETE /identity/v3/projects/d733f5d350a84d0da303204b4067c78e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:34.118233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd8e0-303d-4759-94fd-fe44fc0fa07d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.118545 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd8e0-303d-4759-94fd-fe44fc0fa07d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.119013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd8e0-303d-4759-94fd-fe44fc0fa07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.139928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-267bd8e0-303d-4759-94fd-fe44fc0fa07d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:34.141299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-267bd8e0-303d-4759-94fd-fe44fc0fa07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.176724 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:34.185582 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.186848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.187389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.188410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] PATH_INFO: `/v3/users/f58a59c389cf4b3198a6c452eb2772d6` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.190261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58a59c389cf4b3198a6c452eb2772d6)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.198449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.225592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1303/2614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:34.228198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.228514 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.228847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.239586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:34.254925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:34.256059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58a59c389cf4b3198a6c452eb2772d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.256645 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.263771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58a59c389cf4b3198a6c452eb2772d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.265454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1304/2615] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:34.269741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.276831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58a59c389cf4b3198a6c452eb2772d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.277572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58a59c389cf4b3198a6c452eb2772d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.278809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9ea65-8afb-4709-a357-eaafa04fd852 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.284371 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.285136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.285544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.285822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.286350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58a59c389cf4b3198a6c452eb2772d6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.286471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:34.286843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.288661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.292383 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1312/2616] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:34 2022] DELETE /identity/v3/users/f58a59c389cf4b3198a6c452eb2772d6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:34.303659 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1305/2617] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:34.307944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.309379 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.389083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0d216-c290-4e4a-a842-c2b1132af6a0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.394837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.395468 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.395749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.396021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.396663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.398604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.426982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1313/2618] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:34.431046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.442105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a140c1-9fe7-4dfa-872c-2e8507e3b4e7 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.489339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.491118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.507237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.508211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.508211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.508584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.509313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.511391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.535859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1306/2619] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:34.539682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.550907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47354c3-0648-44f8-b9d9-fc6b535e5eed admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.555679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.556290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.556577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.556854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.557499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.559130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.570599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1307/2620] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:34.574181 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0aced-347d-4b28-a0cd-7b425102d116 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.574423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.579172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.580038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.580326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.581243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] PATH_INFO: `/v3/projects/7225740627444e3a97ff730460c52e7d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.582617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7225740627444e3a97ff730460c52e7d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.584200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eae48-c263-4691-8797-8f6d16dd7c87 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.585201 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.587722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:34.588899 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.589668 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.589967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.590311 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.590932 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.592679 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.603381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1308/2621] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:34.607863 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.619500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fec92-4e80-41a0-b89e-70f4640153ff admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.629997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.630686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.630965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.631237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] PATH_INFO: `/v3/projects/ff1cfb24c84c4a9eb195cbf75ae5de38/users/3ad8106a2015424a99a0f77b9d4c9fd0/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.639596 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7225740627444e3a97ff730460c52e7d', 'request_id': 'req-d72cd292-ab04-4b78-a315-5be8f9227868'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.646152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7225740627444e3a97ff730460c52e7d', 'request_id': 'req-d72cd292-ab04-4b78-a315-5be8f9227868'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.647694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1cfb24c84c4a9eb195cbf75ae5de38, user_id=3ad8106a2015424a99a0f77b9d4c9fd0, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.650391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a2e94-5073-411c-915f-262d4aacbcb6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.653424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7225740627444e3a97ff730460c52e7d', 'request_id': 'req-d72cd292-ab04-4b78-a315-5be8f9227868'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.665365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1309/2622] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:34 2022] PUT /identity/v3/projects/ff1cfb24c84c4a9eb195cbf75ae5de38/users/3ad8106a2015424a99a0f77b9d4c9fd0/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:34.666950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] The token cache is being invalidate because project 7225740627444e3a97ff730460c52e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:34.667243 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.667885 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-d72cd292-ab04-4b78-a315-5be8f9227868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:34.669358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1314/2623] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:34 2022] DELETE /identity/v3/projects/7225740627444e3a97ff730460c52e7d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:34.675071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde543b6-71e5-4569-a7e8-04d49cd31fa3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.675358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde543b6-71e5-4569-a7e8-04d49cd31fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.675633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde543b6-71e5-4569-a7e8-04d49cd31fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.675807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.676109 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.676364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.696255 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-cde543b6-71e5-4569-a7e8-04d49cd31fa3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:34.697473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cde543b6-71e5-4569-a7e8-04d49cd31fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.704758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] MFA Rules not processed for user `3ad8106a2015424a99a0f77b9d4c9fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:34.705929 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.760834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1310/2624] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:34.768811 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.769093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.769992 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.771805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1315/2625] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:34.776190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.777331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.792154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d931d-4125-4265-b05d-0675865d92fe None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.793484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:34.794721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:34.797178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.798436 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.798779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.799040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.799668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.801541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.804851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1311/2626] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:34.808990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.823104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca966177-6215-4e29-886e-ae36f0c01200 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.828781 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.829608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.829897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.830177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.830880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:34.831190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.832357 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1316/2627] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:34.832794 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.836656 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.849681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1312/2628] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:05:34.850651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.861918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e2d7d-b7c1-45d3-a65d-4ee43bfab29d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.866450 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:34.903844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991244c-ba65-4f85-b8d1-09d4851c0c62 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.909245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.909848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.910129 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.910402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.911060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.912960 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.921127 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:34.923310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.924263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.924672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.925073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.926008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.929464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.940852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1317/2629] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:34.942101 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.952239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d275b-0145-4ac8-92c2-6ac5688aac39 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:34.957624 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:34.958268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:34.958578 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:34.958852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:34.959528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:34.961096 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:34.963870 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1313/2630] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:34.968195 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.974855 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1318/2631] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:34.978727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:34.989236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576cc8-a3e7-4f7a-a8ea-6787e0f404db admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.035193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dab5-c08f-4c22-81ed-65098c629669 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.039808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.040412 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.040688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.040955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.041576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.043517 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.046112 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.046753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.047033 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.047937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.048604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.050195 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.060294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1319/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.064423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.066915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1314/2633] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:34 2022] POST /identity/v3/users => generated 279 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:35.070550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.074836 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3d97f-691b-431d-b743-4deb3f5c22c3 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.079689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.080332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.080611 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.080726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa478-96f1-4ebf-9ae6-7d51d79eead9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.080887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] PATH_INFO: `/v3/projects/e0cd52749e0a4f67b9e518d4f4deed7a/users/2a79d649471b481da2b9de590e6f1b0a/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.086297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.087026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.087440 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.087730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.088435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.090388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.097589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0cd52749e0a4f67b9e518d4f4deed7a, user_id=2a79d649471b481da2b9de590e6f1b0a, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.100305 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483fe12-fbed-47eb-b0a5-3c3953833085 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.101135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1315/2634] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.105325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.116443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6bbf4-f50e-4d5b-abaa-f842517d5ecd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.119416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1320/2635] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/e0cd52749e0a4f67b9e518d4f4deed7a/users/2a79d649471b481da2b9de590e6f1b0a/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.131216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.131496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.131765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.159595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] MFA Rules not processed for user `2a79d649471b481da2b9de590e6f1b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:35.160803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:35.175271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.175883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.176163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.176437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.177115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.178732 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.188881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1316/2636] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.193328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.202977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876763-9693-4327-b311-1852b5f60aa0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.208382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.209023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.209303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.209576 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] PATH_INFO: `/v3/projects/96b87aa323e041c18ce1d91c168bc932/users/29bb8f595871442388fdec3ae963dc33/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.212702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:35.229096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b87aa323e041c18ce1d91c168bc932, user_id=29bb8f595871442388fdec3ae963dc33, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.231036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1321/2637] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:35.231817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b771b6-df2c-4792-bdb5-490b886c4632 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.235426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.245494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1317/2638] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/96b87aa323e041c18ce1d91c168bc932/users/29bb8f595871442388fdec3ae963dc33/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.246327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922d013-f95e-4d67-abe4-0b2fc6b934e0 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.256902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.257184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.257457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.284657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:35.286841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] MFA Rules not processed for user `29bb8f595871442388fdec3ae963dc33`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:35.288161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:35.304056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.304665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.304942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.305212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.305833 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.307742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.335685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1322/2639] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:35.339444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.357122 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1318/2640] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:35.361858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.373533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39322fb-e3a0-4a7d-82bf-bd853fafea7a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.423860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f25b9-7be3-4027-9357-7757b9e133eb admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.428583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.429194 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.429475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.429748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.430378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.432318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.440303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.441565 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.442233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.442692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.443611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.445998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.455814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1323/2641] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:35.459801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.471057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016042-6ffc-4ec0-b8be-88474f6dc35d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.479240 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.479948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.480293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.480623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.481278 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.486207 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1319/2642] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/projects => generated 357 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:35.486366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.487606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.497557 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1324/2643] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.498554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.509145 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc28e6-2925-4876-b651-2fe0f40bc862 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.552616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25dde0d-1ebc-47be-abca-f20f9ea084e5 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.557926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.558558 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.558846 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.559144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.559764 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.561677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.566619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.567275 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.567552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.567825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] PATH_INFO: `/v3/projects/f82647f54e9a4217972ba70dabf3c0dd/users/59d6551952ad4ba287dff9590c688c1b/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.570159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:35.582983 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82647f54e9a4217972ba70dabf3c0dd, user_id=59d6551952ad4ba287dff9590c688c1b, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.585376 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1320/2644] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:35.585601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.589590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.599946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1325/2645] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/f82647f54e9a4217972ba70dabf3c0dd/users/59d6551952ad4ba287dff9590c688c1b/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.601051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60163-65e2-453b-8f07-9feb5a2d362b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.603756 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.633262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:35.657901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.658522 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.658813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.659082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.659721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.661313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.668229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cbe05-e99d-4b6d-acd4-16e7917375a6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.670961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1321/2646] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.673583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.674186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.674460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.674758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.675678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.676075 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.677246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.685722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0001-8bb0-4a87-b118-d869c45a436c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.687215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1326/2647] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.691589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.691723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.692319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.692713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.692999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] PATH_INFO: `/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4/users/3e6c4f6a641e41f98c34807c756361be/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.701568 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93a9b0-dafe-421a-b96a-0d536c0fc3c6 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.706430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.707000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.707276 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.707545 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] PATH_INFO: `/v3/projects/f82647f54e9a4217972ba70dabf3c0dd/users/59d6551952ad4ba287dff9590c688c1b/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.708383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8410d0926a42b7945c9f503bf0a8d4, user_id=3e6c4f6a641e41f98c34807c756361be, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.709466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82647f54e9a4217972ba70dabf3c0dd, user_id=59d6551952ad4ba287dff9590c688c1b, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.711059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd1f14-8d54-4337-85ce-f89d53671d04 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.712124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.723382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1327/2648] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/f82647f54e9a4217972ba70dabf3c0dd/users/59d6551952ad4ba287dff9590c688c1b/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.725358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1322/2649] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4/users/3e6c4f6a641e41f98c34807c756361be/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.731258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.733596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.733949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.734162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.763535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] MFA Rules not processed for user `59d6551952ad4ba287dff9590c688c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:35.764759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:35.798712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf1291-ed4c-46a7-ba29-526b3e2d9636 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.835608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1323/2650] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:35.847759 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.849189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:35.862010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.862666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.863000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.863280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.863932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.865896 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.877606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1328/2651] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:35.885766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:35.900544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f907b-399e-4dff-83f4-a72affb6aae7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.905683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.906442 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.906785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.907265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] PATH_INFO: `/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4/users/3e6c4f6a641e41f98c34807c756361be/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.909067 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:35.909406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8410d0926a42b7945c9f503bf0a8d4, user_id=3e6c4f6a641e41f98c34807c756361be, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.912212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e166a-05ab-411b-87c7-e248ba8d2876 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.914382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d2052-7e58-4c49-8fcd-e018e7489a4d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:35.919055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:35.919657 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.919937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.920233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.924016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1329/2652] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:35 2022] PUT /identity/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4/users/3e6c4f6a641e41f98c34807c756361be/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:35.930071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:35.931536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:35.934077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:35.934367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:35.934717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:35.941862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1324/2653] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:35.963772 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] MFA Rules not processed for user `3e6c4f6a641e41f98c34807c756361be`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:35.964991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:36.000613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:36.042741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1330/2654] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:36.054798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:36.056295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:36.068063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e78e02-e001-4846-9241-141910aa69f8 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:36.128848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:36.129464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:36.129741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:36.130012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:36.139490 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:36.140956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:36.151521 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1325/2655] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:36.231584 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:36.241607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e6f8e-fca8-4389-b2a6-f9f3746f4c64 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:36.245797 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:36.246553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:36.247307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:36.247643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:36.308616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:36.310246 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:36.320769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1331/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:36.485833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:36.495178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b5a5-889e-4039-8a45-488e86627e4c service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:36.498962 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:36.499557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:36.499835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:36.500105 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:36.558189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:36.559675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a31cd-d01a-4f77-bf84-aaf28e14378d service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:36.569536 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1326/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:38.309435 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:38.311920 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:38.381738 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d35a2-8f5f-4f2f-845a-de95b8a21508 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:38.387175 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:38.388146 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:38.388478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:38.388766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] PATH_INFO: `/v3/users/5f08cf0be8ae42d79d5e5a33f81620c6` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:38.389912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f08cf0be8ae42d79d5e5a33f81620c6)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:38.392064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:38.437858 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f08cf0be8ae42d79d5e5a33f81620c6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:38.445935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f08cf0be8ae42d79d5e5a33f81620c6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:38.453919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:38.459270 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f08cf0be8ae42d79d5e5a33f81620c6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:38.460010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f08cf0be8ae42d79d5e5a33f81620c6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:38.468009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f08cf0be8ae42d79d5e5a33f81620c6'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:38.474277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1332/2658] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:38 2022] DELETE /identity/v3/users/5f08cf0be8ae42d79d5e5a33f81620c6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:38.478809 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:38.479271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:38.479370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:38.499199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:38.500432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:38.560295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1327/2659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:38.570450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:38.571863 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:38.628322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a4b65-8f51-413d-8cc4-4ac2db1c0a84 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:38.631846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:38.632446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:38.632719 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:38.632986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:38.642225 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:38.643690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536f310-547e-4b8f-9ce2-69d9a05d1e68 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:38.653378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1333/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:38.944950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:38.946495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:38.981177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36542152-23a2-4b56-82c1-6b6a3b64a10a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:38.994656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:38.995400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:38.995748 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:38.996117 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] PATH_INFO: `/v3/projects/180f271dc4d04009a2ddd31a982b429a` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:38.998604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180f271dc4d04009a2ddd31a982b429a)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.002135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.050072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:39.062727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180f271dc4d04009a2ddd31a982b429a', 'request_id': 'req-9497f570-413a-40f9-9f3f-734de509a8b3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:39.072171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180f271dc4d04009a2ddd31a982b429a', 'request_id': 'req-9497f570-413a-40f9-9f3f-734de509a8b3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:39.078189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180f271dc4d04009a2ddd31a982b429a', 'request_id': 'req-9497f570-413a-40f9-9f3f-734de509a8b3'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:39.091466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] The token cache is being invalidate because project 180f271dc4d04009a2ddd31a982b429a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:39.091762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:39.092413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-9497f570-413a-40f9-9f3f-734de509a8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:39.093902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1328/2661] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:38 2022] DELETE /identity/v3/projects/180f271dc4d04009a2ddd31a982b429a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:39.098825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.099120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.099397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.121059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:39.122252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:39.183268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1334/2662] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:39.191474 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.192016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.192289 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.212672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:39.213876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:39.222257 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1329/2663] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:39.226918 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.237089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7e8aa-c96d-411e-a2e9-145deccd46f8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.241924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.242600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.242880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.243208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] PATH_INFO: `/v3/domains` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.243852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:39.244152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.245779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.259077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1335/2664] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:05:39.263906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.273793 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da18bf2-abe5-464a-81b5-8ba31da86c64 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.278750 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.279356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.279634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.279904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.280525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.283411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.311648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1330/2665] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:39.315735 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.328182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9b9a3-0d57-482d-8019-e830af4351db admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.405245 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.406001 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.406399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.406685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.408113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.410289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.432283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1336/2666] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:39.436769 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.447177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91e0b-3f15-4ed6-8f3b-4196feadc666 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.451817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.452418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.452696 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.452965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.453608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.455309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.465043 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1331/2667] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:39.468825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.474488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:39.480878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce518e18-ff1e-4176-9688-24ebd78632ea admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.487069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.487667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.487949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.488293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.488864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.490473 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da212480-12f6-40de-8d49-f4035b9ca1b7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.502257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1337/2668] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:39.506446 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.516847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a48da-b62b-4060-99c3-0de436c8a7fd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.521465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.522107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.522386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.522692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] PATH_INFO: `/v3/projects/69163bf3afca4624ac424104060dfee7/users/e63bef905671411b9124f568e981d5ae/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.537537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69163bf3afca4624ac424104060dfee7, user_id=e63bef905671411b9124f568e981d5ae, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.540220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.553466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1332/2669] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:39 2022] PUT /identity/v3/projects/69163bf3afca4624ac424104060dfee7/users/e63bef905671411b9124f568e981d5ae/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:39.564509 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.564794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.565067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.593049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] MFA Rules not processed for user `e63bef905671411b9124f568e981d5ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:39.594283 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:39.659444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1338/2670] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:39.663852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.726261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60ac2f-3803-48aa-9d5e-b2594f020239 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.730880 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.731481 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.731758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.732029 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.732650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.734575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.760695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1333/2671] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:39.764813 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.775984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46b445-055b-4671-95f4-609c44043de7 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.833407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.834014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.834326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.834667 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.835447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.838077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.861521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1339/2672] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:39.865805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.877353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71beb053-8472-49a4-8d70-89db0764b7b3 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.882719 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.883330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.883611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.883886 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.884530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.886136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.897489 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1334/2673] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:39.901519 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.912160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ca2e1-4284-4baa-a989-76d0a6303dfd admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.916749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.917339 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.917636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.918123 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.918805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.920388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb2d07-3a39-4577-8c8f-de01505a2ba9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.930310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1340/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:39.934366 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:39.944527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bc7e5-c655-4add-b719-5ed5f3092c29 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:39.949138 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:39.949779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.950059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.950331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] PATH_INFO: `/v3/projects/844bbc00e4534b64b5eaf989e08f70fa/users/91cbd35eaacd450aad599b5dc060cc60/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:39.965230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=844bbc00e4534b64b5eaf989e08f70fa, user_id=91cbd35eaacd450aad599b5dc060cc60, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:39.967913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:39.977281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:39.982323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1335/2675] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:39 2022] PUT /identity/v3/projects/844bbc00e4534b64b5eaf989e08f70fa/users/91cbd35eaacd450aad599b5dc060cc60/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:39.993298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:39.993686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:39.994015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.022569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] MFA Rules not processed for user `91cbd35eaacd450aad599b5dc060cc60`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:40.023777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:40.070954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:40.095750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1341/2676] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:40.100379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.165157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf12261-aff4-41ce-b1c6-7a9e9a6b4d2c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.169956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.170594 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.170876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.171144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.171780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.173956 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.205168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1336/2677] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:40 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:40.209183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.219973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ab824-3d6f-4944-ab8f-4feeedeaf56c None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.280031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.280804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.281174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.281543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.282672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.284762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.305867 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1342/2678] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:40 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:40.310073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.322699 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d328199-15e5-4bf1-9b7c-a33888e1dc4b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.327550 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.328171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.328455 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.328726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.330399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.332070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.343147 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1337/2679] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:40.347848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.358489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a11512-9ab4-436f-8144-d2c64c1915b8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.363928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.364804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.365091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.365369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] PATH_INFO: `/v3/projects/c3857a671d584f8e9a37ec39d047fbd0/users/c79801178f8549ceaf88933489b444bb/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.383686 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3857a671d584f8e9a37ec39d047fbd0, user_id=c79801178f8549ceaf88933489b444bb, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.386455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.403403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1343/2680] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:40 2022] PUT /identity/v3/projects/c3857a671d584f8e9a37ec39d047fbd0/users/c79801178f8549ceaf88933489b444bb/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:40.404661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.416862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153dffd-161f-4b9e-aa79-24fd2a0b22c2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.484671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.485862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.486428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.486796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.487592 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.489449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf4732-4a76-44a0-bbfe-6206cf218150 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.500171 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1338/2681] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:40.504680 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.526835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0f77c-0d33-43d5-98eb-b7b4f9b397d7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.526835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.526835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.526835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.527257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] PATH_INFO: `/v3/projects/c3857a671d584f8e9a37ec39d047fbd0/users/c79801178f8549ceaf88933489b444bb/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.527257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3857a671d584f8e9a37ec39d047fbd0, user_id=c79801178f8549ceaf88933489b444bb, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.527257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.547258 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1344/2682] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:40 2022] PUT /identity/v3/projects/c3857a671d584f8e9a37ec39d047fbd0/users/c79801178f8549ceaf88933489b444bb/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:40.549777 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.550814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.551252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.587201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] MFA Rules not processed for user `c79801178f8549ceaf88933489b444bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:40.589506 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:40.656242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.659973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:40.665285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:40.677571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1339/2683] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:40.690177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:40.691660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:40.721189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed03bcc-3710-4db1-8f0f-ba367bdd5fd4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.724926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.725530 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.725810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.726082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.727880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:40.750906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a7d2e-63e4-423c-8ae6-58ac482a74e6 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:40.755173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:40.755789 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:40.756073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:40.756345 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:40.789682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:40.794586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.796793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.806623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1345/2684] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:40.820883 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:40.822494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:40.832428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1340/2685] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:42.363219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:42.371661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c6aa-e451-41b6-8eda-58be322acd3f service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:42.375209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:42.375798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:42.376074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:42.376344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:42.432528 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:42.433956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:42.443622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1346/2686] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:42.993862 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:42.995559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:43.103055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afd586-7cd1-4dc8-b12f-85ed4131c059 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:43.117678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:43.119494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.120362 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.120800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] PATH_INFO: `/v3/users/3ad8106a2015424a99a0f77b9d4c9fd0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.122290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad8106a2015424a99a0f77b9d4c9fd0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:43.124638 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:43.126618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:43.135560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e97b-ca50-4ef3-b822-50e9a8c427e7 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:43.140039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:43.140661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.141039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.141414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.167064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:43.169059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805aa3bc-ca65-41af-9e69-8b0655c9890e service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:43.184775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1347/2687] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:43.199417 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad8106a2015424a99a0f77b9d4c9fd0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.204858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:43.215715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad8106a2015424a99a0f77b9d4c9fd0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.229641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad8106a2015424a99a0f77b9d4c9fd0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.244312 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad8106a2015424a99a0f77b9d4c9fd0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.245178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad8106a2015424a99a0f77b9d4c9fd0'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.252092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1341/2688] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:42 2022] DELETE /identity/v3/users/3ad8106a2015424a99a0f77b9d4c9fd0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:43.256734 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.257070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.257348 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.282071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:43.283696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:43.400465 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1348/2689] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:43.414454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:43.417439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:43.489066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729ded1-ab35-4ff1-8b67-4cdaf9ad7639 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:43.492752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:43.493373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.493656 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.493931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.503754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:43.505296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:43.515503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1342/2690] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:43.788268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:43.790387 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:43.805252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9969-e6aa-41ac-8444-6e9bba2f4d57 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:43.810134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:43.810804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.811081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.811349 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] PATH_INFO: `/v3/projects/ff1cfb24c84c4a9eb195cbf75ae5de38` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.812458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1cfb24c84c4a9eb195cbf75ae5de38)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:43.814306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:43.862215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1cfb24c84c4a9eb195cbf75ae5de38', 'request_id': 'req-42914757-b789-45de-b6e0-49cbce376fc1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.867881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1cfb24c84c4a9eb195cbf75ae5de38', 'request_id': 'req-42914757-b789-45de-b6e0-49cbce376fc1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.873977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1cfb24c84c4a9eb195cbf75ae5de38', 'request_id': 'req-42914757-b789-45de-b6e0-49cbce376fc1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.886432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] The token cache is being invalidate because project ff1cfb24c84c4a9eb195cbf75ae5de38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:43.886759 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.887400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:43.888879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1349/2691] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:43 2022] DELETE /identity/v3/projects/ff1cfb24c84c4a9eb195cbf75ae5de38 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:43.892922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:43.903208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758017b-4336-4daf-9b95-a43125ef22ed service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:43.958961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:43.959580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:43.959853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:43.960118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] PATH_INFO: `/v3/users/2a79d649471b481da2b9de590e6f1b0a` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:43.961284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a79d649471b481da2b9de590e6f1b0a)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:43.963299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:44.008030 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a79d649471b481da2b9de590e6f1b0a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.020507 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a79d649471b481da2b9de590e6f1b0a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.027685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a79d649471b481da2b9de590e6f1b0a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.029669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:44.037469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a79d649471b481da2b9de590e6f1b0a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.038293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a79d649471b481da2b9de590e6f1b0a'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.044549 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1343/2692] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:43 2022] DELETE /identity/v3/users/2a79d649471b481da2b9de590e6f1b0a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:44.314113 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:44.316126 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:44.325512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:44.342066 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914757-b789-45de-b6e0-49cbce376fc1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:44.410069 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:44.410878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:44.411094 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:44.411386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] PATH_INFO: `/v3/projects/e0cd52749e0a4f67b9e518d4f4deed7a` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:44.412597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0cd52749e0a4f67b9e518d4f4deed7a)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:44.414503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:44.464375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0cd52749e0a4f67b9e518d4f4deed7a', 'request_id': 'req-00896ab8-e227-4ffe-82fb-9ee698b8c037'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.475443 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0cd52749e0a4f67b9e518d4f4deed7a', 'request_id': 'req-00896ab8-e227-4ffe-82fb-9ee698b8c037'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.482845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0cd52749e0a4f67b9e518d4f4deed7a', 'request_id': 'req-00896ab8-e227-4ffe-82fb-9ee698b8c037'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.498581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] The token cache is being invalidate because project e0cd52749e0a4f67b9e518d4f4deed7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:44.499103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.499947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.501428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1350/2693] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:44 2022] DELETE /identity/v3/projects/e0cd52749e0a4f67b9e518d4f4deed7a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:44.506000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:44.507839 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:44.524593 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6efa4-c341-4b06-b441-114008febc23 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:44.604472 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:44.605293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:44.605646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:44.605996 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] PATH_INFO: `/v3/users/59d6551952ad4ba287dff9590c688c1b` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:44.607249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d6551952ad4ba287dff9590c688c1b)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:44.609328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:44.670841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d6551952ad4ba287dff9590c688c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.685914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d6551952ad4ba287dff9590c688c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.697033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d6551952ad4ba287dff9590c688c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.706843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d6551952ad4ba287dff9590c688c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.707615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-ba7adff3-9856-4280-bc5d-b7854e3db60e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d6551952ad4ba287dff9590c688c1b'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:44.714395 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1344/2694] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:44 2022] DELETE /identity/v3/users/59d6551952ad4ba287dff9590c688c1b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:45.020506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.021982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:45.037333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00896ab8-e227-4ffe-82fb-9ee698b8c037 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.094412 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.095073 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.095350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.095623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] PATH_INFO: `/v3/projects/f82647f54e9a4217972ba70dabf3c0dd` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.097464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82647f54e9a4217972ba70dabf3c0dd)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.099388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.137287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:45.147364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82647f54e9a4217972ba70dabf3c0dd', 'request_id': 'req-3434d626-9391-403b-ac1d-056bac966ebe'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:45.152787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82647f54e9a4217972ba70dabf3c0dd', 'request_id': 'req-3434d626-9391-403b-ac1d-056bac966ebe'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:45.158820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82647f54e9a4217972ba70dabf3c0dd', 'request_id': 'req-3434d626-9391-403b-ac1d-056bac966ebe'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:45.171803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] The token cache is being invalidate because project f82647f54e9a4217972ba70dabf3c0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:45.172098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:45.172717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3434d626-9391-403b-ac1d-056bac966ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:45.174182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1351/2695] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:45 2022] DELETE /identity/v3/projects/f82647f54e9a4217972ba70dabf3c0dd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:45.180137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.180603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.180703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.200630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:45.201817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:45.258919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1345/2696] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:45.268140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.269023 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.269324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.288934 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:45.290132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:45.297924 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1352/2697] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:45.302718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.312487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcc04f-185a-43e1-9e6e-9a6fe3167917 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.318027 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.318652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.318934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.319209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.319848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:45.320151 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.321746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.328439 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:45.334596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1346/2698] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:45.339003 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.348380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf663c-6ee5-4380-8236-2c31879be5ae None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.353048 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.353650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.353928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.354200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.354876 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.356788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.384591 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1353/2699] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:45.388774 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.399614 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff15d9-1ada-45d4-ab07-0d92d3a8b8ec admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.456148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.456757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.457060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.457322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.458016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.460034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.480797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1347/2700] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:45.485088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.496429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a761a66-65e8-4f17-a2f6-fa0573718fec admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.501252 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.501854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.502118 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.502386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.503692 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.505268 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.515234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1354/2701] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:45.519720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.530351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01862-cfc6-4c2a-a510-348b142db497 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.535295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.535894 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.536175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.536447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.537092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.538726 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604835-19d0-45a9-a26d-4ff617c2add0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.550390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1348/2702] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:45.554842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.564988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47cf7e-2939-4b53-bb00-9674911b0ac5 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.570444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.571119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.571399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.571676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] PATH_INFO: `/v3/projects/ccc26bba8bc24bde85a2fb46cb7d5862/users/d04dadf9ab5247279781d21466cbec55/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.586035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc26bba8bc24bde85a2fb46cb7d5862, user_id=d04dadf9ab5247279781d21466cbec55, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.588700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.602503 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1355/2703] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:45 2022] PUT /identity/v3/projects/ccc26bba8bc24bde85a2fb46cb7d5862/users/d04dadf9ab5247279781d21466cbec55/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:45.673924 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.674142 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.674387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.704423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] MFA Rules not processed for user `d04dadf9ab5247279781d21466cbec55`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:45.705635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:45.771707 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1349/2704] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:45.781697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.842944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162b87-6c12-4c13-b3f7-4dca96e41d6f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.848736 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820702df-e584-4abb-8983-658b6777903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.849354 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820702df-e584-4abb-8983-658b6777903d admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.849638 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820702df-e584-4abb-8983-658b6777903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.849919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820702df-e584-4abb-8983-658b6777903d admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.850586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820702df-e584-4abb-8983-658b6777903d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.852515 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820702df-e584-4abb-8983-658b6777903d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.884141 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1356/2705] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:45.885276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:45.896516 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a15a2-2051-4907-95d4-670f9af01ff0 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:45.970667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:45.972056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:45.972494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:45.973226 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:45.973963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:45.976035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:45.988644 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:46.011889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1350/2706] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:45 2022] POST /identity/v3/users => generated 306 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:46.016016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820702df-e584-4abb-8983-658b6777903d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.027571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820702df-e584-4abb-8983-658b6777903d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.032330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.033070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.033413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.033690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.035086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.036798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.056460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1357/2707] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:46.057561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.068394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8cb8-d64c-4c05-a340-56397b34f834 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.074181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.074928 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.075214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.075492 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] PATH_INFO: `/v3/projects/a955de0ffe2a45a497409f5d9c84200b/users/0519b8f4d9494aa2bc133f3b6b9abe7d/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.092539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a955de0ffe2a45a497409f5d9c84200b, user_id=0519b8f4d9494aa2bc133f3b6b9abe7d, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.095238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.108838 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1351/2708] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:46 2022] PUT /identity/v3/projects/a955de0ffe2a45a497409f5d9c84200b/users/0519b8f4d9494aa2bc133f3b6b9abe7d/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:46.113878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.124076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaf554-e5b4-4c48-8c48-98fb110890a0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.157187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:46.183236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.183774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.184054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.184353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.184996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.186592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c86915-cad4-4099-9c7a-62d73f6702e9 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.196984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1358/2709] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:46.201187 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.211698 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbe898-2d30-4ff7-ae03-a84181473378 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.216335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.216978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.217254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.217542 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] PATH_INFO: `/v3/projects/a955de0ffe2a45a497409f5d9c84200b/users/0519b8f4d9494aa2bc133f3b6b9abe7d/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.219499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a955de0ffe2a45a497409f5d9c84200b, user_id=0519b8f4d9494aa2bc133f3b6b9abe7d, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.222131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.232302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1352/2710] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:46 2022] PUT /identity/v3/projects/a955de0ffe2a45a497409f5d9c84200b/users/0519b8f4d9494aa2bc133f3b6b9abe7d/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:46.242662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.242947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.243219 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.271274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] MFA Rules not processed for user `0519b8f4d9494aa2bc133f3b6b9abe7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:46.272470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:46.341551 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1359/2711] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:46 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:46.353438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.355228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:46.420559 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c9b6-4fd7-4d2b-9213-2dcd189c8bf0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.424165 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.424773 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.425047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.425322 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.434950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.436319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.446578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1353/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:46.520938 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:46.530529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9d459-d5a2-4961-b9dd-263e1f6a0473 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:46.534178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:46.534809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:46.535871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:46.536134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:46.591970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:46.593398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48801536-f2db-45a6-9378-ba4658ed094a service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:46.602849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1360/2713] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:47.167561 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:47.169053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:47.234909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445aaeb-8070-44bc-907a-8923a24475bd service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:47.240662 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:47.241422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:47.241780 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:47.242060 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] PATH_INFO: `/v3/users/e63bef905671411b9124f568e981d5ae` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:47.243205 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63bef905671411b9124f568e981d5ae)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:47.245130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:47.295077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63bef905671411b9124f568e981d5ae'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.310604 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:47.312563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63bef905671411b9124f568e981d5ae'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.324485 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63bef905671411b9124f568e981d5ae'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.334209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63bef905671411b9124f568e981d5ae'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.334889 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63bef905671411b9124f568e981d5ae'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.341194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1354/2714] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:47 2022] DELETE /identity/v3/users/e63bef905671411b9124f568e981d5ae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:47.346282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:47.346712 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:47.347083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:47.371469 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:47.374699 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:47.428535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:47.439861 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1361/2715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:47.450470 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:47.451887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:47.509306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74773c8-b372-4431-b126-f411901771d1 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:47.513140 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:47.513841 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:47.514125 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:47.514398 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:47.524531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:47.525902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:47.535378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1355/2716] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:47.801479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:47.803072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:47.814804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79873ded-7388-48e0-b573-ba0e3415f818 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:47.820438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:47.821074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:47.821410 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:47.821642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] PATH_INFO: `/v3/projects/69163bf3afca4624ac424104060dfee7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:47.822793 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69163bf3afca4624ac424104060dfee7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:47.824684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:47.874601 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69163bf3afca4624ac424104060dfee7', 'request_id': 'req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.880879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69163bf3afca4624ac424104060dfee7', 'request_id': 'req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.887061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69163bf3afca4624ac424104060dfee7', 'request_id': 'req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.899947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] The token cache is being invalidate because project 69163bf3afca4624ac424104060dfee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:47.900065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.900745 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:47.902612 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1362/2717] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:47 2022] DELETE /identity/v3/projects/69163bf3afca4624ac424104060dfee7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:47.906299 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:47.916000 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a615-0d0e-43c4-943c-92737678ddbc service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:47.973562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:47.974194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:47.974563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:47.974766 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] PATH_INFO: `/v3/users/91cbd35eaacd450aad599b5dc060cc60` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:47.975898 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cbd35eaacd450aad599b5dc060cc60)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:47.977823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:48.027273 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cbd35eaacd450aad599b5dc060cc60'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.040601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cbd35eaacd450aad599b5dc060cc60'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.048259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cbd35eaacd450aad599b5dc060cc60'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.056375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cbd35eaacd450aad599b5dc060cc60'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.057033 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cbd35eaacd450aad599b5dc060cc60'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.062966 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1356/2718] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:47 2022] DELETE /identity/v3/users/91cbd35eaacd450aad599b5dc060cc60 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:48.377400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:48.379672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:48.402453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85d8ac-cfa2-4a9b-a379-9107adde3b7b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:48.466887 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:48.467599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:48.467942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:48.468301 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] PATH_INFO: `/v3/projects/844bbc00e4534b64b5eaf989e08f70fa` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:48.469455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=844bbc00e4534b64b5eaf989e08f70fa)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:48.471375 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:48.520081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844bbc00e4534b64b5eaf989e08f70fa', 'request_id': 'req-b16b4c5f-c41b-48c4-a0db-043d5edda985'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.525744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844bbc00e4534b64b5eaf989e08f70fa', 'request_id': 'req-b16b4c5f-c41b-48c4-a0db-043d5edda985'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.532149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844bbc00e4534b64b5eaf989e08f70fa', 'request_id': 'req-b16b4c5f-c41b-48c4-a0db-043d5edda985'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.546647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] The token cache is being invalidate because project 844bbc00e4534b64b5eaf989e08f70fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:48.546937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.547575 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.549051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1363/2719] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:48 2022] DELETE /identity/v3/projects/844bbc00e4534b64b5eaf989e08f70fa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:48.553817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:48.555258 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:48.566616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:48.572993 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88272632-97f3-4307-a838-024079bcac10 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:48.633982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:48.634653 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:48.634934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:48.635209 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] PATH_INFO: `/v3/users/c79801178f8549ceaf88933489b444bb` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:48.636357 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79801178f8549ceaf88933489b444bb)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:48.638295 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:48.685129 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79801178f8549ceaf88933489b444bb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.697682 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79801178f8549ceaf88933489b444bb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.704890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79801178f8549ceaf88933489b444bb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.712686 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79801178f8549ceaf88933489b444bb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.713333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b649fe0-309f-4cc9-813e-995f1758f36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79801178f8549ceaf88933489b444bb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:48.719085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1357/2720] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:48 2022] DELETE /identity/v3/users/c79801178f8549ceaf88933489b444bb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:48.990649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:48.992220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:49.011156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b4c5f-c41b-48c4-a0db-043d5edda985 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.016808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:49.070257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.071054 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.071518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.071972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] PATH_INFO: `/v3/projects/c3857a671d584f8e9a37ec39d047fbd0` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.073732 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3857a671d584f8e9a37ec39d047fbd0)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.076839 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.136649 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3857a671d584f8e9a37ec39d047fbd0', 'request_id': 'req-ca721c01-772d-40de-bd46-a264aa888a4e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:49.142179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3857a671d584f8e9a37ec39d047fbd0', 'request_id': 'req-ca721c01-772d-40de-bd46-a264aa888a4e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:49.148444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3857a671d584f8e9a37ec39d047fbd0', 'request_id': 'req-ca721c01-772d-40de-bd46-a264aa888a4e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:49.160780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] The token cache is being invalidate because project c3857a671d584f8e9a37ec39d047fbd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:49.161065 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:49.161681 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca721c01-772d-40de-bd46-a264aa888a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:49.163222 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1364/2721] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:48 2022] DELETE /identity/v3/projects/c3857a671d584f8e9a37ec39d047fbd0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:49.169189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.169475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.169746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.189632 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:49.190847 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:49.249511 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1358/2722] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:49.258217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.258504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.258806 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.278765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:49.279978 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:49.288006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1365/2723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:49.292217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.302337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bf6e-5e2b-4a0a-bbd7-c130790548fd None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.306965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.312309 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.323352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1359/2724] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:49.327456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.337318 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a2d6-a1c2-4a71-80cf-fab852a6da74 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.342114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.342764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.343046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.343398 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.344082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.345981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.378589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1366/2725] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:49 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:49.383502 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.394025 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451993c9-ef54-42f3-abe6-ecaebf6beb4b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.399252 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:49.458458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.459106 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.459388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.459664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.460335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.464386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.487800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1360/2726] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:49 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:49.489860 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.500405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997ec3-da3e-4d88-a5df-1d2d1c80b595 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.505821 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.506424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.506838 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.507077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.507731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.509396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.523662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1367/2727] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:05:49.524681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.535092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea7d-21fb-4306-a7c2-03ad6812d7ff admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.540480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.541282 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.541437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.541648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.542290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.544154 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b3c8e-22d7-455c-bdb8-3eb6198b2646 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.556234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1361/2728] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:49.560674 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.571213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59998e3-3338-4a4c-8811-196de347b6b0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.575913 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.576501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.576779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.577055 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] PATH_INFO: `/v3/projects/aee8a3a6942b46d1acd604181255db4e/users/bfad996a69bb4f20bc0ce67b20fe34fe/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.591697 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee8a3a6942b46d1acd604181255db4e, user_id=bfad996a69bb4f20bc0ce67b20fe34fe, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.594310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.609617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1368/2729] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:49 2022] PUT /identity/v3/projects/aee8a3a6942b46d1acd604181255db4e/users/bfad996a69bb4f20bc0ce67b20fe34fe/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:49.621066 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.621414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.621762 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.653119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] MFA Rules not processed for user `bfad996a69bb4f20bc0ce67b20fe34fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:49.654354 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:49.719672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1362/2730] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:49.732668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:49.734623 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:49.780213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:49.796947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9de417-abd5-4734-8723-87cfa9c4c799 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:49.802009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:49.802892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:49.803171 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:49.803484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:49.812518 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:49.814125 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae7f6c-a7f3-4345-86f3-dad341dad44e service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:49.824726 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1369/2731] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:50.740364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:50.741758 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:50.805478 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b96c38-be7a-4f1b-b8e6-23d61c541b0a None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:50.809919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:50.810631 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:50.810914 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:50.811185 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] PATH_INFO: `/v3/users/d04dadf9ab5247279781d21466cbec55` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:50.812297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04dadf9ab5247279781d21466cbec55)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:50.814194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:50.835484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:50.861113 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04dadf9ab5247279781d21466cbec55'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:50.873197 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04dadf9ab5247279781d21466cbec55'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:50.881261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04dadf9ab5247279781d21466cbec55'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:50.888080 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04dadf9ab5247279781d21466cbec55'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:50.888700 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04dadf9ab5247279781d21466cbec55'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:50.896198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1363/2732] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:50 2022] DELETE /identity/v3/users/d04dadf9ab5247279781d21466cbec55 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:50.900461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:50.900942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:50.901067 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:50.920713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:50.922041 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:50.981511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1370/2733] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:50.992471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:50.993967 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:51.058250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55363ea0-3d58-497a-a56f-9d9ac8c22df2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:51.062822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:51.063586 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:51.064131 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:51.064581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:51.074831 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:51.076310 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:51.087035 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1364/2734] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:51.391889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:51.393668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:51.406799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fdb00-a85d-4403-bec9-67ac522ef823 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:51.414320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:51.415803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:51.416629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:51.416629 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] PATH_INFO: `/v3/projects/ccc26bba8bc24bde85a2fb46cb7d5862` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:51.417663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc26bba8bc24bde85a2fb46cb7d5862)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:51.419617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:51.477274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc26bba8bc24bde85a2fb46cb7d5862', 'request_id': 'req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.483424 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc26bba8bc24bde85a2fb46cb7d5862', 'request_id': 'req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.490143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc26bba8bc24bde85a2fb46cb7d5862', 'request_id': 'req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.509176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] The token cache is being invalidate because project ccc26bba8bc24bde85a2fb46cb7d5862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:51.509330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.510238 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.514748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1371/2735] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:51 2022] DELETE /identity/v3/projects/ccc26bba8bc24bde85a2fb46cb7d5862 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:51.515690 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:51.533358 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822db3b9-6598-4ad6-a5c6-5dfff7b45fd1 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:51.591694 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:51.592361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:51.592800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:51.593119 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] PATH_INFO: `/v3/users/0519b8f4d9494aa2bc133f3b6b9abe7d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:51.594269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0519b8f4d9494aa2bc133f3b6b9abe7d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:51.596245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:51.636637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:51.645822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0519b8f4d9494aa2bc133f3b6b9abe7d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.658965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0519b8f4d9494aa2bc133f3b6b9abe7d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.667855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0519b8f4d9494aa2bc133f3b6b9abe7d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.675004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0519b8f4d9494aa2bc133f3b6b9abe7d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.675636 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-1c0a6f6b-3add-4b9f-a472-94682ac2dc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0519b8f4d9494aa2bc133f3b6b9abe7d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:51.682083 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1365/2736] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:51 2022] DELETE /identity/v3/users/0519b8f4d9494aa2bc133f3b6b9abe7d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:51.963982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:51.965513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:51.981079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56e2d7-9ba3-4bb9-ab90-1252a81330b1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.039322 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.039963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.040247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.040525 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] PATH_INFO: `/v3/projects/a955de0ffe2a45a497409f5d9c84200b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.041657 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a955de0ffe2a45a497409f5d9c84200b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.043606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.045864 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:52.092256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a955de0ffe2a45a497409f5d9c84200b', 'request_id': 'req-50034569-4651-4d87-bb2d-18b2619b140c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.097769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a955de0ffe2a45a497409f5d9c84200b', 'request_id': 'req-50034569-4651-4d87-bb2d-18b2619b140c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.103804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a955de0ffe2a45a497409f5d9c84200b', 'request_id': 'req-50034569-4651-4d87-bb2d-18b2619b140c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.117074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] The token cache is being invalidate because project a955de0ffe2a45a497409f5d9c84200b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:52.117399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.118035 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-50034569-4651-4d87-bb2d-18b2619b140c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.119595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1372/2737] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:51 2022] DELETE /identity/v3/projects/a955de0ffe2a45a497409f5d9c84200b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:52.124609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.125086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.125192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.146807 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:52.148059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.206198 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1366/2738] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:52.215293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.215580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.215852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.235161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:52.236441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.244768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1373/2739] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:52.248842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.259237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7efad-5410-4cef-9c7e-e0845a08a945 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.263785 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.264386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.264671 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.264941 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.265580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:52.265885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.267520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.278903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1367/2740] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:52.283331 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.293114 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8841ff-94b7-4ae5-b7a2-dc01b37eb250 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.297773 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.298367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.298676 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.298949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.299585 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.301981 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.329116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1374/2741] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:52.333281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.343087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f512-9c26-40b6-80c5-157c6d252641 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.399355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.399982 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.400259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.400529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.401159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.403111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.433231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1368/2742] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:52.434541 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.446042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa61e-5db7-4a04-a72f-1563e3faf155 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.451196 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.451892 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.452239 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.452521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.453173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.454782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.465886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1375/2743] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:52.470053 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.481386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573be85-fca8-485b-bec9-b153111ea153 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.486451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.487090 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.487373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.487649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.488294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.489965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913ab1e-1337-496d-a12d-96293ace5c08 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.501334 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1369/2744] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:52.505691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.515914 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444f8-1e93-4e4e-99d9-0fa80cd1a603 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.521359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.522015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.522290 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.522586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] PATH_INFO: `/v3/projects/150c68f1b9924fcbb9ef01321b766c24/users/d86c6bb562da456c8cd7df1e34763e00/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.538293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150c68f1b9924fcbb9ef01321b766c24, user_id=d86c6bb562da456c8cd7df1e34763e00, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.541221 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.558788 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1376/2745] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:52 2022] PUT /identity/v3/projects/150c68f1b9924fcbb9ef01321b766c24/users/d86c6bb562da456c8cd7df1e34763e00/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:52.569370 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.569665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.570031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.600118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] MFA Rules not processed for user `d86c6bb562da456c8cd7df1e34763e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:52.601347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.613779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:52.673557 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1370/2746] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:52.684645 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.689287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.689616 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.691023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.749377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc20a2a-de06-49d1-9911-27f2ce52aca2 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.753068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.753682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.753965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.754317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.759434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f80085-6c59-4dab-8744-22ede50e9ee8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.763150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.764589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.764740 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.765369 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.765655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.765930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] PATH_INFO: `/v3/users/29bb8f595871442388fdec3ae963dc33` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.767086 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bb8f595871442388fdec3ae963dc33)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.769089 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.775293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1377/2747] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:52.795558 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.807139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada33bb-6d7a-4d30-9e72-169300e82256 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:52.812208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:52.812830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.813105 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.813378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] PATH_INFO: `/v3/users/d86c6bb562da456c8cd7df1e34763e00` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.814497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86c6bb562da456c8cd7df1e34763e00)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:52.816478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:52.817949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bb8f595871442388fdec3ae963dc33'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.830829 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bb8f595871442388fdec3ae963dc33'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.840980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bb8f595871442388fdec3ae963dc33'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.849051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bb8f595871442388fdec3ae963dc33'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.849752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-54f227c3-f219-4ad2-88b2-620d17819603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bb8f595871442388fdec3ae963dc33'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.849901 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:52.857822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1371/2748] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:52 2022] DELETE /identity/v3/users/29bb8f595871442388fdec3ae963dc33 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:52.863335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.863676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.863950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.872766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86c6bb562da456c8cd7df1e34763e00'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.880894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86c6bb562da456c8cd7df1e34763e00'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.884011 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:52.885245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.893309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86c6bb562da456c8cd7df1e34763e00'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.894039 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86c6bb562da456c8cd7df1e34763e00'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.901762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a4b66757-b9c5-4418-84a1-2d73b41e2553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86c6bb562da456c8cd7df1e34763e00'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:52.907764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1378/2749] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:52 2022] DELETE /identity/v3/users/d86c6bb562da456c8cd7df1e34763e00 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:52.912161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:52.912444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:52.912713 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:52.915635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:52.932790 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:52.933923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.948378 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1372/2750] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:52.958562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:52.959920 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:52.995970 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1379/2751] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:53.005602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.007040 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.016976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226d09-0fb1-41c1-83ea-fb495c6d02c3 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.064581 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.064916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3433-de3f-4e84-9ce4-23acdb8ee8b5 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.065189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.065531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.065743 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.069513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.070104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.070381 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.070687 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.075736 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.077175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.079909 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.081329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.087613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1373/2752] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:53.092184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1380/2753] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:53.567904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.569984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.578800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.581295 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.585349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed619f6-4f8b-409d-9f28-228847eb475f service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.596300 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.597336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.597336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.598302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] PATH_INFO: `/v3/projects/150c68f1b9924fcbb9ef01321b766c24` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.599697 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=150c68f1b9924fcbb9ef01321b766c24)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.602544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.631951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8691ba2-e30e-48ae-a5af-129c22360ee8 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.639250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.640744 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.641154 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.641471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] PATH_INFO: `/v3/projects/96b87aa323e041c18ce1d91c168bc932` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.643224 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b87aa323e041c18ce1d91c168bc932)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.646588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.664901 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150c68f1b9924fcbb9ef01321b766c24', 'request_id': 'req-481c6abf-dbb8-415e-893b-e553542c39ce'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.673449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150c68f1b9924fcbb9ef01321b766c24', 'request_id': 'req-481c6abf-dbb8-415e-893b-e553542c39ce'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.680094 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150c68f1b9924fcbb9ef01321b766c24', 'request_id': 'req-481c6abf-dbb8-415e-893b-e553542c39ce'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.688212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:53.692678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] The token cache is being invalidate because project 150c68f1b9924fcbb9ef01321b766c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:53.692970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.693887 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-481c6abf-dbb8-415e-893b-e553542c39ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.695647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1374/2754] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:53 2022] DELETE /identity/v3/projects/150c68f1b9924fcbb9ef01321b766c24 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:53.696777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b87aa323e041c18ce1d91c168bc932', 'request_id': 'req-fa503242-fd96-4798-a0d5-0f7aec18dc27'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.700193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd2ab5-42e1-4111-86f8-27a26296b13f None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.700473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd2ab5-42e1-4111-86f8-27a26296b13f None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.700739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd2ab5-42e1-4111-86f8-27a26296b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.706419 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b87aa323e041c18ce1d91c168bc932', 'request_id': 'req-fa503242-fd96-4798-a0d5-0f7aec18dc27'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.713891 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b87aa323e041c18ce1d91c168bc932', 'request_id': 'req-fa503242-fd96-4798-a0d5-0f7aec18dc27'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.721363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-86cd2ab5-42e1-4111-86f8-27a26296b13f None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:53.722567 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-86cd2ab5-42e1-4111-86f8-27a26296b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.726529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] The token cache is being invalidate because project 96b87aa323e041c18ce1d91c168bc932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:53.726849 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.727501 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.729198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1381/2755] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:53 2022] DELETE /identity/v3/projects/96b87aa323e041c18ce1d91c168bc932 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:53.733100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.734333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.735341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:53.789531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1375/2756] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:53.796999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.797347 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.797689 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.800247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa503242-fd96-4798-a0d5-0f7aec18dc27 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.805840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.806498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.806810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.807087 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] PATH_INFO: `/v3/users/3e6c4f6a641e41f98c34807c756361be` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.808359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6c4f6a641e41f98c34807c756361be)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.810539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.817673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:53.819203 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.827246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1376/2757] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:53.831447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.843426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c95ea3-1ba7-4e5d-94e4-a06fd7cb0452 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.849320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.850054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.850335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.850675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.851319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:53.851621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.853211 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.860945 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6c4f6a641e41f98c34807c756361be'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.866843 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1377/2758] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:05:53.868753 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6c4f6a641e41f98c34807c756361be'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.869998 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.880615 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c9cd0-308b-408c-b7a1-dc1b2442489b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:53.881782 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6c4f6a641e41f98c34807c756361be'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.882606 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6c4f6a641e41f98c34807c756361be'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.885461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:53.886225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:53.886504 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:53.886919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:53.887787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:53.890058 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:53.890200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6c4f6a641e41f98c34807c756361be'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:53.898168 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1382/2759] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:53 2022] DELETE /identity/v3/users/3e6c4f6a641e41f98c34807c756361be => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:53.923143 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1378/2760] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:53 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:53.924376 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:53.925846 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:53.981378 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:54.000173 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b56a-cb06-4aa6-bf8f-a380786b23ee admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.070535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.071179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.071516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.071801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] PATH_INFO: `/v3/users` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.072441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.074553 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.101076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1383/2761] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:53 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:54.102426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.114290 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc425a-25c3-468f-9925-2b800ce3a92d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.120721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.121383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.121661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.121935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.122629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.124243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.135219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1379/2762] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:54.139506 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.151636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ab35-f248-4092-a933-4027d7229d64 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.156711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.157364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.157640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.157911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.158599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.160273 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0c2e5-1e30-42c7-802e-d5e4270bb75e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.172262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1384/2763] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:54.177133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.187906 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212eb514-c54f-4c80-a519-267a7b10fe1b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.193539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.194180 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.194461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.194779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] PATH_INFO: `/v3/projects/98f8feff9e134cf6a3af309733aa9bf8/users/bf106dbdcbc64874baea077877a787be/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.211531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f8feff9e134cf6a3af309733aa9bf8, user_id=bf106dbdcbc64874baea077877a787be, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.214335 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.229104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1380/2764] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:54 2022] PUT /identity/v3/projects/98f8feff9e134cf6a3af309733aa9bf8/users/bf106dbdcbc64874baea077877a787be/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:54.240946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.241232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.241511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.272287 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] MFA Rules not processed for user `bf106dbdcbc64874baea077877a787be`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:54.273432 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:54.343119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1385/2765] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:54.357797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.360034 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:54.421466 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57718e-bcdd-449f-8ce2-9b6997352728 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.426288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.427751 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.427928 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:54.428377 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.428655 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.428926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.439078 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.440525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4f675-c4bc-4087-992d-5ac4e338d52a service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.451667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1381/2766] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:54.509817 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3609-1bc0-4963-ae02-ff2adcebd33b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.515574 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.516217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.516495 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.516768 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] PATH_INFO: `/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.517974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8410d0926a42b7945c9f503bf0a8d4)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.520760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.527702 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:54.570668 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8410d0926a42b7945c9f503bf0a8d4', 'request_id': 'req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:54.577467 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8410d0926a42b7945c9f503bf0a8d4', 'request_id': 'req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:54.584546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8410d0926a42b7945c9f503bf0a8d4', 'request_id': 'req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:54.597115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] The token cache is being invalidate because project 4b8410d0926a42b7945c9f503bf0a8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:54.597400 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:54.598156 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-e01ca3ad-cb27-4a8c-98dc-ab839b2bf59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:54.599811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1386/2767] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:54 2022] DELETE /identity/v3/projects/4b8410d0926a42b7945c9f503bf0a8d4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:05:54.605323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.605988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.606274 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.627541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:54.628873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:54.672935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:54.698832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1382/2768] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:54.709527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.710683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.711015 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.732531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:54.733749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:54.742857 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1387/2769] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:54.746770 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.757433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7b015-85d1-4994-9958-4c142a96881d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.762541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.763177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.763457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.763733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.764386 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:05:54.764693 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.766316 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.779152 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1383/2770] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:05:54.783316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.793408 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950afacc-0ff6-4dec-83ee-b8934e62cce6 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.798155 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.798830 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.799116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.799389 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.800032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.802586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.830396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1388/2771] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:54 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:54.835639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.847244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d12e-836a-496f-b7ca-7696f38c28e2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.902946 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.903541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.903822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.904093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.904723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.906727 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.927983 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1384/2772] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:54.932135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.942336 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb0501-513b-45f1-ad3d-74b5ea60f687 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.947426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.948028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.948303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.948571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.949211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.950879 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.961390 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1389/2773] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:54.965179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:54.975622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a08b-1e48-49d7-80a1-d2866ba9f17b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:54.980363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:54.980951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:54.981228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:54.981500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:54.982139 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:54.983752 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9a163-2a55-47d7-92ea-0560fe8a6d6f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:54.994275 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1385/2774] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:54.998481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.008317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe2003-3a24-4650-9d00-66e992146865 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.012932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.013569 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.013845 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.014120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] PATH_INFO: `/v3/projects/f741f6610fe64d808a5cc4a4db6cba71/users/81d62c81b3a84e1fb0671511c7055afc/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.029334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f741f6610fe64d808a5cc4a4db6cba71, user_id=81d62c81b3a84e1fb0671511c7055afc, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.031923 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.045206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1390/2775] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:54 2022] PUT /identity/v3/projects/f741f6610fe64d808a5cc4a4db6cba71/users/81d62c81b3a84e1fb0671511c7055afc/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:55.055296 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.055605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.055877 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.087038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] MFA Rules not processed for user `81d62c81b3a84e1fb0671511c7055afc`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:55.087038 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:55.151225 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1386/2776] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:05:55.155604 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.216215 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe27f80-438b-4c1a-9e20-757b4b99357e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.220769 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.221359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.221635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.221906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.222560 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.224949 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.251208 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1391/2777] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:05:55 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:05:55.255791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.266353 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf68ef-e831-4d1d-b675-1726c677e9f4 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.299344 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:55.324315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.324937 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.325254 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.325562 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.326215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.328218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.349633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1387/2778] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:05:55 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:05:55.353618 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.363733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c3d4-d700-4e15-a609-a5a22b24ef98 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.368682 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.369274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.369550 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.369822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.370478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.372056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.383965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1392/2779] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:55.388189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.398341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7439f7a-3e7b-40a2-9630-85d22d684f54 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.402988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.403650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.403968 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.404287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] PATH_INFO: `/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7/users/367b68d09b7f47e285c5e68f0a7f7892/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.419633 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a12d4ef1c94f35a6f047f1ece1b7a7, user_id=367b68d09b7f47e285c5e68f0a7f7892, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.422436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.435893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1388/2780] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:55 2022] PUT /identity/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7/users/367b68d09b7f47e285c5e68f0a7f7892/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:55.439895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.446544 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:55.450434 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d62477-5331-4214-bf4f-d5663a190ce8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.507438 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.508043 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.508320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.508597 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.509231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.510844 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71d23-76ae-4f61-822f-63b9c40cd117 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.520905 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1393/2781] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:05:55.525590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.536621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20f9e4-b236-4955-beb1-c1a6bda57615 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.541183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.541842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.542160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.542456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] PATH_INFO: `/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7/users/367b68d09b7f47e285c5e68f0a7f7892/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.544475 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a12d4ef1c94f35a6f047f1ece1b7a7, user_id=367b68d09b7f47e285c5e68f0a7f7892, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.547224 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.557223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1389/2782] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:05:55 2022] PUT /identity/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7/users/367b68d09b7f47e285c5e68f0a7f7892/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:55.568516 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.568800 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.569076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.596309 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] MFA Rules not processed for user `367b68d09b7f47e285c5e68f0a7f7892`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:55.597500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:55.668701 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1394/2783] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:55.679714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.681451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:55.739404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b963eb-1a13-47ea-86a6-67407490d117 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.743026 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.743624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.743903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.744178 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.753063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.754493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.764156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1390/2784] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:55.879253 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:55.889531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287e56-5921-47b7-8fd8-042cd64b9047 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:55.893174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:55.893778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:55.894062 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:55.894341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:55.955959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:55.957402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4e6aa-ee44-43bb-aaa5-dd096cd1b85b service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:55.968176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1395/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:05:59.013190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:59.014749 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:59.079552 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf8832-4c1b-4640-b225-f7600d7f2f51 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:59.084746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:59.085375 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:59.085652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:59.085922 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] PATH_INFO: `/v3/users/bfad996a69bb4f20bc0ce67b20fe34fe` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:59.087059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfad996a69bb4f20bc0ce67b20fe34fe)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:59.088978 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:59.128237 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:59.139757 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfad996a69bb4f20bc0ce67b20fe34fe'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.152348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfad996a69bb4f20bc0ce67b20fe34fe'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.159499 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfad996a69bb4f20bc0ce67b20fe34fe'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.166613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfad996a69bb4f20bc0ce67b20fe34fe'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.167228 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfad996a69bb4f20bc0ce67b20fe34fe'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.172950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1391/2786] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:05:59 2022] DELETE /identity/v3/users/bfad996a69bb4f20bc0ce67b20fe34fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:05:59.178009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:59.178576 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:59.178747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:59.198097 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:05:59.199321 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:59.217477 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:05:59.258489 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1396/2787] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:05:59.268085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:59.269494 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:59.327634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61252c-6ba4-45b8-a320-055d9675bcb9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:59.331302 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:59.331902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:59.332181 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:59.332454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:59.341661 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:59.343115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:59.352695 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1392/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:05:59.618742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:05:59.620182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:05:59.640718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553090d0-0b19-419f-ae6a-561f0a6dcad3 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:05:59.646758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:05:59.647652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:05:59.648050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:05:59.648429 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] PATH_INFO: `/v3/projects/aee8a3a6942b46d1acd604181255db4e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:05:59.649705 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee8a3a6942b46d1acd604181255db4e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:05:59.652422 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:05:59.711341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee8a3a6942b46d1acd604181255db4e', 'request_id': 'req-f543b6b2-38d8-4817-a7aa-41ea540e9f10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.717464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee8a3a6942b46d1acd604181255db4e', 'request_id': 'req-f543b6b2-38d8-4817-a7aa-41ea540e9f10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.723780 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee8a3a6942b46d1acd604181255db4e', 'request_id': 'req-f543b6b2-38d8-4817-a7aa-41ea540e9f10'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.735725 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] The token cache is being invalidate because project aee8a3a6942b46d1acd604181255db4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:05:59.736016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.736635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-f543b6b2-38d8-4817-a7aa-41ea540e9f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:05:59.738107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1397/2789] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:05:59 2022] DELETE /identity/v3/projects/aee8a3a6942b46d1acd604181255db4e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:03.863722 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:03.865909 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:03.878635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010a63-ff72-49dc-b0f2-db52e7315f4f service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:03.934418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:03.935069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:03.935348 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:03.935618 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] PATH_INFO: `/v3/users/81d62c81b3a84e1fb0671511c7055afc` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:03.936741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d62c81b3a84e1fb0671511c7055afc)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:03.938702 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:03.984272 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d62c81b3a84e1fb0671511c7055afc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:03.991900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:03.998135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d62c81b3a84e1fb0671511c7055afc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.006540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d62c81b3a84e1fb0671511c7055afc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.013832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d62c81b3a84e1fb0671511c7055afc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.014476 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d62c81b3a84e1fb0671511c7055afc'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.020277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1393/2790] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:03 2022] DELETE /identity/v3/users/81d62c81b3a84e1fb0671511c7055afc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:04.024771 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:04.025256 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:04.025369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:04.044989 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:04.046172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:04.105985 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1398/2791] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:04.116276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:04.117703 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:04.179818 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739e1c3-c483-4f5f-aa58-1a8defc9ac56 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:04.183493 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:04.184097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:04.184379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:04.184650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:04.193868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:04.195603 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:04.204895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1394/2792] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:04.482098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:04.483588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:04.496176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002087bb-3a4b-4266-a7ea-5d77c819233b None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:04.501095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:04.501932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:04.502209 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:04.502488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] PATH_INFO: `/v3/projects/f741f6610fe64d808a5cc4a4db6cba71` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:04.503709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f741f6610fe64d808a5cc4a4db6cba71)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:04.505663 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:04.557599 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:04.563459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f741f6610fe64d808a5cc4a4db6cba71', 'request_id': 'req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.572886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f741f6610fe64d808a5cc4a4db6cba71', 'request_id': 'req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.580021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f741f6610fe64d808a5cc4a4db6cba71', 'request_id': 'req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.594077 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] The token cache is being invalidate because project f741f6610fe64d808a5cc4a4db6cba71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:04.594372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.595034 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.596717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1399/2793] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:04 2022] DELETE /identity/v3/projects/f741f6610fe64d808a5cc4a4db6cba71 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:04.601042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:04.610741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a663-a63b-488a-88fe-2638a62f11c0 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:04.670352 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:04.671036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:04.671320 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:04.671591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] PATH_INFO: `/v3/users/367b68d09b7f47e285c5e68f0a7f7892` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:04.672685 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367b68d09b7f47e285c5e68f0a7f7892)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:04.674627 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:04.721230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367b68d09b7f47e285c5e68f0a7f7892'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.734041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367b68d09b7f47e285c5e68f0a7f7892'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.741169 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367b68d09b7f47e285c5e68f0a7f7892'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.746556 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:04.748884 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367b68d09b7f47e285c5e68f0a7f7892'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.749670 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-8a240df1-e729-4e7a-a46d-68d225477bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367b68d09b7f47e285c5e68f0a7f7892'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:04.757093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1395/2794] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:04 2022] DELETE /identity/v3/users/367b68d09b7f47e285c5e68f0a7f7892 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:06:05.038426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.039944 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:05.055017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbf2f7-a415-4d8f-a8c3-f6fefdb8e4b7 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.116072 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.116662 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.116947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.117226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] PATH_INFO: `/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.118371 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a12d4ef1c94f35a6f047f1ece1b7a7)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.120369 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.168095 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a12d4ef1c94f35a6f047f1ece1b7a7', 'request_id': 'req-bdb931ad-22be-43f2-9331-ff919893513e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:05.173825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a12d4ef1c94f35a6f047f1ece1b7a7', 'request_id': 'req-bdb931ad-22be-43f2-9331-ff919893513e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:05.180894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a12d4ef1c94f35a6f047f1ece1b7a7', 'request_id': 'req-bdb931ad-22be-43f2-9331-ff919893513e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:05.193963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] The token cache is being invalidate because project e0a12d4ef1c94f35a6f047f1ece1b7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:05.194653 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:05.195280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bdb931ad-22be-43f2-9331-ff919893513e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:05.196774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1400/2795] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:05 2022] DELETE /identity/v3/projects/e0a12d4ef1c94f35a6f047f1ece1b7a7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:05.202526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21779-1429-4708-ac3f-88a00810792b None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.203018 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21779-1429-4708-ac3f-88a00810792b None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.203136 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21779-1429-4708-ac3f-88a00810792b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.342401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-68d21779-1429-4708-ac3f-88a00810792b None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:05.343652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-68d21779-1429-4708-ac3f-88a00810792b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:05.402469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1396/2796] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:05.410386 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.410711 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.411021 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.432647 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:05.433871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:05.441688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1401/2797] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:05.447118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d21779-1429-4708-ac3f-88a00810792b None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.456692 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d21779-1429-4708-ac3f-88a00810792b None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.461844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.462434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.462746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.463015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.463821 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:06:05.463933 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.465503 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.478940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1397/2798] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 12:06:05.483669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.492964 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce5c1a-b6e0-4c92-a48f-fdbdab1b058e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.497571 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.498165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.498445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.498748 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.499411 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.501282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.528182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1402/2799] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:06:05.532330 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.542988 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d52b66-9231-4230-bdbf-ceda71b7853f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.598479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.599266 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.599554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.599824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.600443 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.602382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.623799 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1398/2800] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:05.627842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.638706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3bdd-d5ff-48b0-b0ce-595fa5be815f admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.643317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.643908 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.644186 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.644460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.645098 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.647428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.657249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1403/2801] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:05.661036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.671510 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0fe2-e82c-4c69-b8d9-fea9ea8a8f37 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.676193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.676708 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.677015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.677313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.677936 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.679535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda578b8-8cfb-42a3-a44a-b42a54719b2c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.689600 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1399/2802] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:05.694783 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.704840 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349456ef-c504-408a-be40-961f8d7fb576 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.710133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.710812 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.711090 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.711365 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] PATH_INFO: `/v3/projects/99c37428b2a54db7bee5bc8cd3926c3c/users/5e3e1759217a48b880fbaa320ee69885/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.726721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c37428b2a54db7bee5bc8cd3926c3c, user_id=5e3e1759217a48b880fbaa320ee69885, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.729394 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.738228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:05.753010 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1404/2803] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:05 2022] PUT /identity/v3/projects/99c37428b2a54db7bee5bc8cd3926c3c/users/5e3e1759217a48b880fbaa320ee69885/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:05.756778 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.757069 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.757434 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.785673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] MFA Rules not processed for user `5e3e1759217a48b880fbaa320ee69885`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:05.787166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:05.836051 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:05.854349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1400/2804] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:05.858968 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.920446 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0355af-b5bb-412a-9176-c98bcd4ac43d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:05.924907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:05.925498 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:05.925775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:05.926051 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:05.926708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:05.929414 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:05.955329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1405/2805] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:05.959919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:05.970713 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13434a76-bf0d-449d-bc05-aaa10e205b72 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.027008 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.027612 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.027891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.028161 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.028797 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.030731 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.050860 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1401/2806] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:05 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:06:06.055757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:06.065965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe3f77-29e1-4d1c-95d6-8e53e42eea28 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.071559 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.072192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.072475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.072755 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.073448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.075169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.086910 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1406/2807] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:06.090705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:06.101533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb31f-2a04-4b7b-8b37-cb3e69433de2 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.106132 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.106808 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.107093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.107373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] PATH_INFO: `/v3/projects/a49bc7508b3b44c788217660543e9f9f/users/b3714cbfe0bc483a8129fd780cde6062/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.122451 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49bc7508b3b44c788217660543e9f9f, user_id=b3714cbfe0bc483a8129fd780cde6062, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.125103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.138382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1402/2808] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:06 2022] PUT /identity/v3/projects/a49bc7508b3b44c788217660543e9f9f/users/b3714cbfe0bc483a8129fd780cde6062/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:06.142406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:06.153187 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0913f9-767a-4aac-b74d-74c4441f9551 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.209700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.210229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.210524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.210820 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.211458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.213047 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cc295-a5b1-44fa-9b6c-0d3e25386d39 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.223265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1407/2809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:06.228394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:06.238878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df6804-76ed-4883-9b8a-d60419a477be admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.243573 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.244244 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.244527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.244824 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] PATH_INFO: `/v3/projects/a49bc7508b3b44c788217660543e9f9f/users/b3714cbfe0bc483a8129fd780cde6062/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.246803 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49bc7508b3b44c788217660543e9f9f, user_id=b3714cbfe0bc483a8129fd780cde6062, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.249424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.259513 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1403/2810] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:06 2022] PUT /identity/v3/projects/a49bc7508b3b44c788217660543e9f9f/users/b3714cbfe0bc483a8129fd780cde6062/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:06.271199 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.271487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.271764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.300948 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] MFA Rules not processed for user `b3714cbfe0bc483a8129fd780cde6062`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:06.302267 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:06.360061 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:06.371814 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1408/2811] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:06.383301 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:06.384971 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:06.440220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298b6c6-9690-4e6e-ae6e-d488835c8d8c admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:06.444605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:06.445199 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:06.445473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:06.445739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:06.501544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:06.503072 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:06.506669 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:06.513438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1404/2812] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:16.166405 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:16.167880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:16.202054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:16.211298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd9aa9-b981-4e49-b3f1-034fb3dd5605 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:16.214935 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:16.215537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:16.215815 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:16.216092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:16.225233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:16.226683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:16.236108 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1405/2813] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:06:16.239179 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493e734-da61-46b0-9589-504cdd0d5313 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:16.244397 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:16.245107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:16.245388 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:16.245678 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:16.247613 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:16.260406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:16.261958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:16.272998 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1409/2814] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:17.719319 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:17.720873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:17.797123 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5492-bc55-42ba-bed4-b365c77231e0 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:17.802338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:17.803547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:17.803857 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:17.804068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:17.805856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:17.821625 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:17.823193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:17.833961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1406/2815] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:18.142334 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:18.154057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78e15-3449-414e-98fe-9e5ade11e7cd service placement] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:18.159404 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:18.160005 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:18.160286 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:18.160617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:18.163184 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:18.174996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:18.176572 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4d147-2423-41eb-abe1-7985bfbc0a3f service placement] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:18.187294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1410/2816] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:19.062059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:19.063596 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.131673 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740237f-41df-4a05-a535-a78cd1bb2baa service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:19.136913 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:19.137540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.137820 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.138092 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] PATH_INFO: `/v3/users/bf106dbdcbc64874baea077877a787be` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.139240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf106dbdcbc64874baea077877a787be)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:19.141219 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:19.187351 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf106dbdcbc64874baea077877a787be'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.201590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf106dbdcbc64874baea077877a787be'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.211399 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf106dbdcbc64874baea077877a787be'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.219848 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf106dbdcbc64874baea077877a787be'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.222869 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf106dbdcbc64874baea077877a787be'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.230064 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1407/2817] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:19 2022] DELETE /identity/v3/users/bf106dbdcbc64874baea077877a787be => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:06:19.234366 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.235828 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.237016 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.260279 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:19.261696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.340742 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1411/2818] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:19.351716 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:19.353294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.369833 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:19.439938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc712b94-2d1a-4389-9c96-0ebe679284b3 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:19.444115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:19.444717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.444986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.445250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.456184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:19.457816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7a961-304d-4906-aaea-bd731b28837c service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:19.469997 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1408/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:06:19.746079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:19.747513 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.759973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4e8f-599d-465d-829a-26fcdd2c0026 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:19.764700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:19.765341 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.765617 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.765889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] PATH_INFO: `/v3/projects/98f8feff9e134cf6a3af309733aa9bf8` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.767046 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f8feff9e134cf6a3af309733aa9bf8)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:19.768942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:19.816317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f8feff9e134cf6a3af309733aa9bf8', 'request_id': 'req-c27f18e4-c8c0-4272-a0fe-394f54b14163'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.822102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f8feff9e134cf6a3af309733aa9bf8', 'request_id': 'req-c27f18e4-c8c0-4272-a0fe-394f54b14163'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.828363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f8feff9e134cf6a3af309733aa9bf8', 'request_id': 'req-c27f18e4-c8c0-4272-a0fe-394f54b14163'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.830917 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:19.845372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] The token cache is being invalidate because project 98f8feff9e134cf6a3af309733aa9bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:19.845661 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.846380 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-c27f18e4-c8c0-4272-a0fe-394f54b14163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:19.847972 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1412/2820] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:19 2022] DELETE /identity/v3/projects/98f8feff9e134cf6a3af309733aa9bf8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:19.853763 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.854054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.854328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.873313 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:19.874554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.947191 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1409/2821] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:19.955200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:19.955486 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:19.955763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:19.977359 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:19.978593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:19.986694 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1413/2822] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:19.991021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.001526 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbc0af-db30-4f4e-a588-6559b6135442 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.007260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.007872 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.008665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.008665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.009036 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:06:20.009332 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.010957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.023885 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1410/2823] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:06:20.028729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.038377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7fdce-41f0-4797-be66-221d17ebd049 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.043539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.044169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.044447 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.044722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.045351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.047296 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.079474 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1414/2824] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:20 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:20.088606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.102432 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3870b-ea22-4659-a46a-0c4ada84832a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.177144 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.177950 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.178409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.178705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.179349 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.181954 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.207240 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1411/2825] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:20 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:20.211852 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.222853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03297b30-7a81-4ca0-b8fa-68f28c11a696 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.228227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.228848 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.229128 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.229402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.230068 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.231733 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.243292 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1415/2826] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:20.247520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.258580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e4be6-a320-458e-802c-e09672fe55fb admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.263591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.264356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.264642 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.264918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.265569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.267212 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87370b26-2f31-4e53-b716-59b6a8cb6b2e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.282156 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1412/2827] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:20.283427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.296110 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffcd7c-c204-4548-b6d6-0067217af00e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.301481 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.302231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.302600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.302956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] PATH_INFO: `/v3/projects/b3df59bc28534911b091999aa03aca2e/users/12b69e54d58048b99559368c0cc9d7cf/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.322916 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3df59bc28534911b091999aa03aca2e, user_id=12b69e54d58048b99559368c0cc9d7cf, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.325658 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.340189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1416/2828] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:20 2022] PUT /identity/v3/projects/b3df59bc28534911b091999aa03aca2e/users/12b69e54d58048b99559368c0cc9d7cf/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:20.345303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.345578 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.345951 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.365583 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:20.366814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:20.375960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:20.425972 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1413/2829] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:20.435660 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:20.437500 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:20.501952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a45bc7-8075-472d-9c80-1e8fe4e050b8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:20.505762 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:20.506449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:20.506827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:20.507103 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:20.517399 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:20.518886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf97460-73be-48b0-811a-edc078940f00 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:20.530167 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1417/2830] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:21.399298 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:21.401634 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:21.415980 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2caa2-8858-434c-9214-78f2bf6626d2 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:21.420952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:21.421591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:21.421861 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:21.422331 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] PATH_INFO: `/v3/users/5e3e1759217a48b880fbaa320ee69885` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:21.423551 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3e1759217a48b880fbaa320ee69885)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:21.425463 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:21.481717 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3e1759217a48b880fbaa320ee69885'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:21.497192 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3e1759217a48b880fbaa320ee69885'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:21.507952 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3e1759217a48b880fbaa320ee69885'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:21.516654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3e1759217a48b880fbaa320ee69885'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:21.517414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3e1759217a48b880fbaa320ee69885'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:21.524433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1414/2831] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:21 2022] DELETE /identity/v3/users/5e3e1759217a48b880fbaa320ee69885 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:21.528691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:21.529190 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:21.529307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:21.549532 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:21.550799 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:21.622973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1418/2832] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:21.635098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:21.636681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:21.757006 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccefa-03bc-41ff-8c1f-2f6ed831f909 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:21.763042 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:21.764084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:21.764084 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:21.764505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:21.783630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:21.786149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:21.799372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1415/2833] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:22.263333 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:22.264731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:22.277962 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569423f6-6e25-4d9b-9878-d25fee4584b8 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:22.283547 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:22.284259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:22.284543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:22.284822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] PATH_INFO: `/v3/projects/99c37428b2a54db7bee5bc8cd3926c3c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:22.286100 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c37428b2a54db7bee5bc8cd3926c3c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:22.289352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:22.325471 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:22.344463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c37428b2a54db7bee5bc8cd3926c3c', 'request_id': 'req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.351079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c37428b2a54db7bee5bc8cd3926c3c', 'request_id': 'req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.357986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c37428b2a54db7bee5bc8cd3926c3c', 'request_id': 'req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.370182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] The token cache is being invalidate because project 99c37428b2a54db7bee5bc8cd3926c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:22.370475 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.371259 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.372984 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1419/2834] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:22 2022] DELETE /identity/v3/projects/99c37428b2a54db7bee5bc8cd3926c3c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:22.376767 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:22.388553 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf7b6-b7d4-417c-b9d0-7bd918cbf1cb service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:22.421073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:22.452902 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:22.453539 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:22.453817 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:22.454091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] PATH_INFO: `/v3/users/b3714cbfe0bc483a8129fd780cde6062` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:22.455246 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3714cbfe0bc483a8129fd780cde6062)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:22.457176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:22.508427 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3714cbfe0bc483a8129fd780cde6062'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.521177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3714cbfe0bc483a8129fd780cde6062'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.531103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3714cbfe0bc483a8129fd780cde6062'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.540448 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3714cbfe0bc483a8129fd780cde6062'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.541261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-18dd92c2-592a-464f-ad82-7f29ffd21b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3714cbfe0bc483a8129fd780cde6062'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:22.549788 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1416/2835] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:22 2022] DELETE /identity/v3/users/b3714cbfe0bc483a8129fd780cde6062 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:22.952053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:22.953600 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:22.966854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb9fd-14d3-4ccd-90ba-ca3ede1274ad admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.043691 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.044497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.044880 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.045280 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] PATH_INFO: `/v3/projects/a49bc7508b3b44c788217660543e9f9f` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.046672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49bc7508b3b44c788217660543e9f9f)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.048940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.142589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49bc7508b3b44c788217660543e9f9f', 'request_id': 'req-93578092-e736-4210-bed4-d4a3e65e63ae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:23.151717 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49bc7508b3b44c788217660543e9f9f', 'request_id': 'req-93578092-e736-4210-bed4-d4a3e65e63ae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:23.168488 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49bc7508b3b44c788217660543e9f9f', 'request_id': 'req-93578092-e736-4210-bed4-d4a3e65e63ae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:23.207957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] The token cache is being invalidate because project a49bc7508b3b44c788217660543e9f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:23.208304 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:23.209941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-93578092-e736-4210-bed4-d4a3e65e63ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:23.211886 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1420/2836] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:22 2022] DELETE /identity/v3/projects/a49bc7508b3b44c788217660543e9f9f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:23.217876 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.218164 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.218495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.253428 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:23.259744 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:23.371646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1417/2837] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:23.379643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.379936 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.380210 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.415133 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:23.417463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:23.434391 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1421/2838] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:06:23.438465 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.452021 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1850af-f3d8-4d93-801a-5ce790d60e00 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.460584 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.461363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.461718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.462081 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.463283 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:06:23.463845 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.465621 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.485085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.491973 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1418/2839] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:06:23.496947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae82497-fc96-4069-8de0-729b10085d87 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.502877 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.504220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.504895 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.505385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.507807 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.510760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.551036 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1422/2840] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:06:23.555814 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.567388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183f856-2a94-4e10-ac25-f1f652b7f5a9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.619688 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:23.651622 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.652338 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.654204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.654605 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.655500 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.658221 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.690637 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1419/2841] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:23.695211 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.710752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcbb7-e750-49a5-b735-1b5e725fd901 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.716926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.718031 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.718377 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.718741 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.720581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.722396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.735294 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1423/2842] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:23.739943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.751606 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d69925-97cf-435d-8ec3-1ac9c4803364 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.756426 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.757111 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.757462 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.757810 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.758541 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.760265 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114365fa-fa4b-40fd-b9f3-651ec1d3d865 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.771796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1420/2843] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:23.775947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:23.787143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2f9b3-532e-4af2-b147-c35b6e5fcb2b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:23.794646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:23.795423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.796487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.796775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] PATH_INFO: `/v3/projects/893fd75baf0544bca2e5cc5aa6258984/users/72979ff82a8e4afb803e168ee80740ee/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.799282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:23.814825 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=893fd75baf0544bca2e5cc5aa6258984, user_id=72979ff82a8e4afb803e168ee80740ee, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:23.817478 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:23.836950 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1424/2844] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:23 2022] PUT /identity/v3/projects/893fd75baf0544bca2e5cc5aa6258984/users/72979ff82a8e4afb803e168ee80740ee/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:06:23.850858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:23.851548 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:23.851895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:23.885390 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] MFA Rules not processed for user `72979ff82a8e4afb803e168ee80740ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:23.886544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:23.968650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1421/2845] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:23.973237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.052721 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a02d-3e59-4247-a2b9-4d23fc791f29 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.058161 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.058801 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.059082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.059626 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.059994 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.061965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.113271 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1425/2846] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:23 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:06:24.117816 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.134194 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5525b-446a-4fd4-ab47-afe7299bd166 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.208050 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.208733 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.209019 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.210115 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.211159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.216261 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.267361 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1422/2847] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:24 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:06:24.271207 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.285815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab58940-1fad-4de5-ad8b-adae9ac8dbb4 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.292758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.293640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.294083 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.294504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.295930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.298140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.313372 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1426/2848] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:24.317647 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.329124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c9a6-02c0-4ab9-9d03-20f842c4980a admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.334382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.335325 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.335786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.336227 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] PATH_INFO: `/v3/projects/6e22bca297b54bf9970352f18fddd982/users/b19b6e4477d54ec7b40ed6a5b538f457/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.358367 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e22bca297b54bf9970352f18fddd982, user_id=b19b6e4477d54ec7b40ed6a5b538f457, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.364454 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.395232 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1423/2849] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:24 2022] PUT /identity/v3/projects/6e22bca297b54bf9970352f18fddd982/users/b19b6e4477d54ec7b40ed6a5b538f457/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:24.396871 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.411749 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321362dd-de5a-4923-b938-f2c559eab369 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.484143 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.484837 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.485076 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.485351 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.486140 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.487835 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a352e-a8e5-404a-a215-f49e503fc076 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.501696 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1427/2850] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:06:24.505897 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.518336 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef25334-e12f-4dc0-b573-d4095c664663 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.524965 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.527435 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.527725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.527999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] PATH_INFO: `/v3/projects/6e22bca297b54bf9970352f18fddd982/users/b19b6e4477d54ec7b40ed6a5b538f457/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.530650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e22bca297b54bf9970352f18fddd982, user_id=b19b6e4477d54ec7b40ed6a5b538f457, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.533023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.541571 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:24.552040 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1424/2851] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:24 2022] PUT /identity/v3/projects/6e22bca297b54bf9970352f18fddd982/users/b19b6e4477d54ec7b40ed6a5b538f457/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:06:24.563579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.563870 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.564152 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.597573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] MFA Rules not processed for user `b19b6e4477d54ec7b40ed6a5b538f457`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:24.599020 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:24.646453 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:24.690264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1428/2852] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:24.702910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.704599 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:24.762590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:24.764242 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:24.785091 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590d69-bb8d-4338-aac5-4b736122e7cc admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.789851 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.790630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.791015 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.791413 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.853402 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25dbf1-d79f-42be-a61d-be2676db6a46 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:24.858298 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:24.859028 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:24.859312 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:24.859592 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:24.861650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:24.871994 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:24.873624 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af33f1-44e5-42b4-a1d4-871cb245f276 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:24.889855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1425/2853] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:25.048659 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.050264 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.080878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.081243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.081602 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.081764 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1429/2854] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:06:25.139384 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] MFA Rules not processed for user `12b69e54d58048b99559368c0cc9d7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:25.140609 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:25.230190 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1426/2855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:25.235150 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.236329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:25.348778 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397b940-1406-422b-bf59-efa44e5bbc91 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.354045 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.354787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.355049 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.355314 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.356247 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.358695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.369644 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=100 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:25.407690 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1430/2856] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:25 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:25.408619 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.455337 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8d87-15f1-412a-b23c-58561e571121 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.518591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=100 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:25.538844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.539509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.539798 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.540074 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.540927 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.543071 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.570791 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1427/2857] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:25 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:25.575353 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.588641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355aabc9-0d6e-46c3-9065-6ada3e6b7343 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.595017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.596038 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.596512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.596787 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.597455 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.599337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.616907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.617593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1431/2858] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:25.627934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1b63-35cf-4160-8753-45b41cf2be55 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.634168 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.634382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.634533 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.634830 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] PATH_INFO: `/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027/users/922b47f760de41d491eba3e43f0f2baf/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.653431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea63c75c6ac248d0b63c05f19fa2b027, user_id=922b47f760de41d491eba3e43f0f2baf, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.656184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.676782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1428/2859] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:25 2022] PUT /identity/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027/users/922b47f760de41d491eba3e43f0f2baf/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:25.681977 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.693459 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0660fca-c9b0-4d20-95d3-32da7a0332d0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.776216 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.776640 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.776930 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.777303 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.777958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.779650 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.793261 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1432/2860] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:25.798245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:25.814217 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871de47-da21-435e-b053-72379fb9b3e4 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:25.822984 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:25.823641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:25.823926 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:25.824355 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] PATH_INFO: `/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027/users/922b47f760de41d491eba3e43f0f2baf/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:25.826639 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea63c75c6ac248d0b63c05f19fa2b027, user_id=922b47f760de41d491eba3e43f0f2baf, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:25.829294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:25.842271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1429/2861] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:25 2022] PUT /identity/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027/users/922b47f760de41d491eba3e43f0f2baf/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:26.612535 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:26.616183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:26.737335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fea02-3e45-43fc-aeee-612a4b3f9682 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:26.862032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e45de636b0e4f35aa279718cc129923', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:26.862761 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:26.863120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:26.863457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:26.865511 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:26.885524 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:26.887428 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2587673-acbf-4f05-967e-38e504c789ad service nova] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:26.893496 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:26.904521 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1433/2862] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:27.895107 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:28.032076 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108df0-0238-46b6-9444-041bea1b6cd0 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:28.041891 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:28.042910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:28.043447 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:28.044509 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] PATH_INFO: `/v3/users/72979ff82a8e4afb803e168ee80740ee` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:28.045730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72979ff82a8e4afb803e168ee80740ee)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:28.048720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:28.144782 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72979ff82a8e4afb803e168ee80740ee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:28.177005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72979ff82a8e4afb803e168ee80740ee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:28.195096 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72979ff82a8e4afb803e168ee80740ee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:28.214001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72979ff82a8e4afb803e168ee80740ee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:28.214364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72979ff82a8e4afb803e168ee80740ee'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:28.228365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1430/2863] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:27 2022] DELETE /identity/v3/users/72979ff82a8e4afb803e168ee80740ee => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:28.233463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:28.234329 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:28.234893 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:28.261396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:28.262684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:28.442919 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1434/2864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:28.462646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:28.464160 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:28.499097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:28.609918 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9960acf-e20a-4238-9a90-450b608a2e95 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:28.614650 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:28.615306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:28.615590 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:28.615904 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:28.657175 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:28.658677 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:28.686990 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1431/2865] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:29.154457 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:29.157552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:29.184499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8abd6-9aa1-4d0b-886a-3f0a64ae0611 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:29.190170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:29.191262 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:29.191737 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:29.192189 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] PATH_INFO: `/v3/projects/893fd75baf0544bca2e5cc5aa6258984` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:29.193959 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=893fd75baf0544bca2e5cc5aa6258984)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:29.196160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:29.342260 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '893fd75baf0544bca2e5cc5aa6258984', 'request_id': 'req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.360747 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '893fd75baf0544bca2e5cc5aa6258984', 'request_id': 'req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.375251 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '893fd75baf0544bca2e5cc5aa6258984', 'request_id': 'req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.392153 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] The token cache is being invalidate because project 893fd75baf0544bca2e5cc5aa6258984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:29.393183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.394237 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.396497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1435/2866] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:29 2022] DELETE /identity/v3/projects/893fd75baf0544bca2e5cc5aa6258984 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:29.400923 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:29.418085 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc7eb5-8860-4364-bccb-a9a63134717c service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:29.585986 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:29.586801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:29.587284 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:29.587651 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] PATH_INFO: `/v3/users/b19b6e4477d54ec7b40ed6a5b538f457` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:29.588977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19b6e4477d54ec7b40ed6a5b538f457)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:29.591387 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:29.697315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19b6e4477d54ec7b40ed6a5b538f457'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.737456 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19b6e4477d54ec7b40ed6a5b538f457'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.762486 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19b6e4477d54ec7b40ed6a5b538f457'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.780014 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19b6e4477d54ec7b40ed6a5b538f457'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.782173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f0c432cb-c0b4-4df1-ac5c-b0813ae3e758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19b6e4477d54ec7b40ed6a5b538f457'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:29.801787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1432/2867] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:29 2022] DELETE /identity/v3/users/b19b6e4477d54ec7b40ed6a5b538f457 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:30.812942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:30.814407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:30.836766 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f56f8-aad8-4f15-b6e7-294471cc24a1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:31.014689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:31.015512 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:31.015804 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:31.016084 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] PATH_INFO: `/v3/projects/6e22bca297b54bf9970352f18fddd982` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:31.019058 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e22bca297b54bf9970352f18fddd982)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:31.023979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:31.097942 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:31.150263 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e22bca297b54bf9970352f18fddd982', 'request_id': 'req-ed76f9dd-9434-4a63-85e1-626853db2f40'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:31.166760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e22bca297b54bf9970352f18fddd982', 'request_id': 'req-ed76f9dd-9434-4a63-85e1-626853db2f40'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:31.189071 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e22bca297b54bf9970352f18fddd982', 'request_id': 'req-ed76f9dd-9434-4a63-85e1-626853db2f40'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:31.213326 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] The token cache is being invalidate because project 6e22bca297b54bf9970352f18fddd982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:31.213965 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:31.216192 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ed76f9dd-9434-4a63-85e1-626853db2f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:31.220079 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1436/2868] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:30 2022] DELETE /identity/v3/projects/6e22bca297b54bf9970352f18fddd982 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:31.224574 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:31.225070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:31.225725 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:31.292532 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:31.298873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:31.376097 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:31.449253 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1433/2869] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:31.458174 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:31.458466 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:31.458763 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:31.507974 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:31.511479 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:31.525619 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1437/2870] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:31.529635 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:31.553437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd008841-df63-467e-8043-cd1ebf509d5d None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:31.567411 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:31.568103 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:31.572388 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:31.572718 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:31.576057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:06:31.576372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:31.581186 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:31.619012 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1434/2871] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:06:31.623075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:31.640607 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1ed8d-616d-42bb-bfee-0f4d2c34a965 None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:31.665135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:31.665688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:31.665966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:31.666236 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:31.667843 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:31.669810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:31.807850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1438/2872] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:06:31 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:31.809291 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:31.837303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df161cbb-b29e-4344-ae2e-c32af4d5cbe9 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:32.002589 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:32.003429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.003715 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.003989 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.004925 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:32.009057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:32.056271 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1435/2873] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:06:31 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:06:32.056963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:32.072487 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492df18a-cfc5-426a-adf7-2ec494163591 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:32.083311 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:32.094641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.095539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.096250 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.097074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:32.099227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:32.112198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1439/2874] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:06:32.120099 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:32.131963 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f53b4-25cf-4e88-a2f7-66e5e949c762 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:32.137333 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:32.138555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.138555 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.138823 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.139579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:32.141231 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a4e43-91aa-4deb-8083-1210c5703b68 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:32.157957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:32.168303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1436/2875] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:06:32.170967 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd84b2e-6bbe-4c56-bb93-ce198e932455 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:32.176823 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:32.177654 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.177939 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.178217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] PATH_INFO: `/v3/projects/02c76e3dce5f41ec83e1382cdf40c3db/users/f92fb5e26a7746d781ccfc3646196cfb/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.202134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c76e3dce5f41ec83e1382cdf40c3db, user_id=f92fb5e26a7746d781ccfc3646196cfb, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:32.204889 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:32.222695 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1440/2876] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:06:32 2022] PUT /identity/v3/projects/02c76e3dce5f41ec83e1382cdf40c3db/users/f92fb5e26a7746d781ccfc3646196cfb/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:06:32.306601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9f809-9bd4-40f1-a5ab-5c988484a694 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.306601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9f809-9bd4-40f1-a5ab-5c988484a694 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.306601 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9f809-9bd4-40f1-a5ab-5c988484a694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.356720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5ec9f809-9bd4-40f1-a5ab-5c988484a694 None None] MFA Rules not processed for user `f92fb5e26a7746d781ccfc3646196cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:32.358234 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5ec9f809-9bd4-40f1-a5ab-5c988484a694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:32.490401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1437/2877] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:32.503811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:32.509996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:32.584546 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d253218-dc74-46a9-b317-8ab55d36d01a admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:32.591567 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:32.592344 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.592684 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.593319 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.602815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:32.604529 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:32.615317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1441/2878] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:32.904890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4a0b-54fe-43a5-be56-6af9255df456 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:32.905176 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4a0b-54fe-43a5-be56-6af9255df456 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:32.905461 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4a0b-54fe-43a5-be56-6af9255df456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:32.954939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-0b8f4a0b-54fe-43a5-be56-6af9255df456 None None] MFA Rules not processed for user `922b47f760de41d491eba3e43f0f2baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:32.956802 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-0b8f4a0b-54fe-43a5-be56-6af9255df456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:32.982985 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:33.107082 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1438/2879] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:33.134991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:33.137075 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:33.210050 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912c668-015e-47a0-8711-3976d2414d63 service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:33.214132 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a59e7ff79249eaa7f45cdbe15d0eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:33.214881 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:33.215223 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:33.215562 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:33.217493 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:33.223403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:33.290911 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:33.292464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:33.301579 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1442/2880] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:33.447262 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dbcec-1abf-4e46-8a60-d5ca1cbbc59a None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:33.447874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dbcec-1abf-4e46-8a60-d5ca1cbbc59a None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:33.447874 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dbcec-1abf-4e46-8a60-d5ca1cbbc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:33.473859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-e95dbcec-1abf-4e46-8a60-d5ca1cbbc59a None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:33.476216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e95dbcec-1abf-4e46-8a60-d5ca1cbbc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:33.542438 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1439/2881] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:06:33.592396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:33.593966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:33.685282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29459a-8e1b-433d-bd3e-516466e3a70e service glance] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:33.691456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:33.692082 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:33.692368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:33.692641 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:33.702418 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:33.704165 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c95b7-0a88-453b-b594-b6ae60949c73 service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:33.714722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1443/2882] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:06:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:06:43.534712 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=45 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:43.709642 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=62 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:43.983879 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:43.984162 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:43.984436 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:44.003917 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] MFA Rules not processed for user `12b69e54d58048b99559368c0cc9d7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:44.005229 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:44.012704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1440/2883] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:06:46.375213 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:46.375754 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:46.375946 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:46.397794 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:06:46.399688 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:46.407441 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1444/2884] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Sep 12 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 12:06:46.423188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:46.435929 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b972d-960d-4e9c-b045-835f7d0e69d8 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:46.441023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:46.441960 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:46.442382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:46.443063 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:46.454445 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:46.456173 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:46.465248 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1441/2885] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:06:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:06:49.431460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:49.433499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:49.517784 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6cd1b-88b2-4a85-8c90-6540554e59db None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:49.523628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:49.524430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:49.524798 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:49.525178 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:49.527135 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:49.615941 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:49.617842 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:49.628714 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1445/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:06:56.461842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=9 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:56.777739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:56.779328 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:56.794049 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63611c3f-1ffe-4180-88dc-03eddd48b2fe service swift] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:56.798687 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:56.799326 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:56.799591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:56.799868 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] PATH_INFO: `/v3/users/fb4bb9cd41cd46ddb87a746e732a948f` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:56.800977 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4bb9cd41cd46ddb87a746e732a948f)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:56.802930 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:56.850382 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4bb9cd41cd46ddb87a746e732a948f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:56.862915 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4bb9cd41cd46ddb87a746e732a948f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:56.871216 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4bb9cd41cd46ddb87a746e732a948f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:56.878433 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4bb9cd41cd46ddb87a746e732a948f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:56.879001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4bb9cd41cd46ddb87a746e732a948f'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:56.884852 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1442/2887] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:56 2022] DELETE /identity/v3/users/fb4bb9cd41cd46ddb87a746e732a948f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:57.146330 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:57.157440 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a88783-8de0-49c3-851e-7694da043a69 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:57.216352 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:57.216991 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:57.217274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:57.217552 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] PATH_INFO: `/v3/projects/6d05fbbec08d43468068ce162f427eeb` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:57.218706 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d05fbbec08d43468068ce162f427eeb)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:57.220614 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:57.263451 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:57.266802 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d05fbbec08d43468068ce162f427eeb', 'request_id': 'req-54801222-c09b-4244-9557-2cabbffa5dc0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.272853 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d05fbbec08d43468068ce162f427eeb', 'request_id': 'req-54801222-c09b-4244-9557-2cabbffa5dc0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.279953 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d05fbbec08d43468068ce162f427eeb', 'request_id': 'req-54801222-c09b-4244-9557-2cabbffa5dc0'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.293009 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] The token cache is being invalidate because project 6d05fbbec08d43468068ce162f427eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:57.293302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.293940 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.295460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1446/2888] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:57 2022] DELETE /identity/v3/projects/6d05fbbec08d43468068ce162f427eeb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:57.299458 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:57.300893 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:57.314276 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7da8-3552-4632-a84d-6427e686a62e admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:57.370540 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:57.371184 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:57.371459 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:57.371728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] PATH_INFO: `/v3/users/8626dd696906488d8d950e6a82ff7235` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:57.385056 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8626dd696906488d8d950e6a82ff7235)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:57.387023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:57.440208 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8626dd696906488d8d950e6a82ff7235'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.458323 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:06:57.459701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8626dd696906488d8d950e6a82ff7235'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.475145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8626dd696906488d8d950e6a82ff7235'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.493297 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8626dd696906488d8d950e6a82ff7235'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.494796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8626dd696906488d8d950e6a82ff7235'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.504193 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1443/2889] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:06:57 2022] DELETE /identity/v3/users/8626dd696906488d8d950e6a82ff7235 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:06:57.765218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:06:57.766727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:06:57.805628 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801222-c09b-4244-9557-2cabbffa5dc0 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:06:57.898149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:06:57.899032 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:06:57.900718 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:06:57.901002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] PATH_INFO: `/v3/projects/ef9b309814494508879ab2ab7b1d629c` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:06:57.912288 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9b309814494508879ab2ab7b1d629c)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:06:57.914188 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:06:57.983091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9b309814494508879ab2ab7b1d629c', 'request_id': 'req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.989324 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9b309814494508879ab2ab7b1d629c', 'request_id': 'req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:57.996730 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9b309814494508879ab2ab7b1d629c', 'request_id': 'req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:58.028445 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] The token cache is being invalidate because project ef9b309814494508879ab2ab7b1d629c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:06:58.028727 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:58.029602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-bfd51587-b2fa-4f1e-a2a7-78321a5fdcae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:06:58.031302 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1447/2890] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:06:57 2022] DELETE /identity/v3/projects/ef9b309814494508879ab2ab7b1d629c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:01.612314 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:01.613723 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:01.627204 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ca7-9dc4-4845-bf9e-02720069acbf admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:01.696949 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:01.697579 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:01.697855 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:01.698124 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] PATH_INFO: `/v3/users/f92fb5e26a7746d781ccfc3646196cfb` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:01.699505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92fb5e26a7746d781ccfc3646196cfb)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:01.701401 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:01.746424 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92fb5e26a7746d781ccfc3646196cfb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:01.758126 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92fb5e26a7746d781ccfc3646196cfb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:01.766269 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92fb5e26a7746d781ccfc3646196cfb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:01.774315 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92fb5e26a7746d781ccfc3646196cfb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:01.774976 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92fb5e26a7746d781ccfc3646196cfb'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:01.781287 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1444/2891] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:07:01 2022] DELETE /identity/v3/users/f92fb5e26a7746d781ccfc3646196cfb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:01.786014 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:01.786573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:01.786703 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:01.806119 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:01.807367 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:01.861274 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:01.874982 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1448/2892] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:01.885238 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:01.886681 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:01.946308 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21127e-3caa-4cea-9f89-a58a2cc6788f admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:01.950159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:01.950796 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:01.951073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:01.951341 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:01.960087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:01.962391 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c246b-35ee-41bc-b1da-00a6f6505f75 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:01.971683 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1445/2893] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:07:02.243957 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.245458 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:02.258176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3486f6-3d34-41dc-8650-c6de7437d26d None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.263760 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.264403 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.264683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.264955 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] PATH_INFO: `/v3/projects/02c76e3dce5f41ec83e1382cdf40c3db` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.266070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c76e3dce5f41ec83e1382cdf40c3db)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.268000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.316822 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c76e3dce5f41ec83e1382cdf40c3db', 'request_id': 'req-0e7ec830-ba0d-4376-8204-be65ffc0592f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:02.322954 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c76e3dce5f41ec83e1382cdf40c3db', 'request_id': 'req-0e7ec830-ba0d-4376-8204-be65ffc0592f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:02.329433 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c76e3dce5f41ec83e1382cdf40c3db', 'request_id': 'req-0e7ec830-ba0d-4376-8204-be65ffc0592f'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:02.341935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] The token cache is being invalidate because project 02c76e3dce5f41ec83e1382cdf40c3db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:07:02.342226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:02.342868 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-0e7ec830-ba0d-4376-8204-be65ffc0592f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:02.344416 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1449/2894] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:07:02 2022] DELETE /identity/v3/projects/02c76e3dce5f41ec83e1382cdf40c3db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:02.349720 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.350007 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.350277 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.357873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:02.372418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:02.373608 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:02.431059 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1446/2895] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:02.438470 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.438776 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.439052 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.459335 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:02.460517 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:02.468456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1450/2896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:02.472629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.482529 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b786c64-8c55-485a-b425-4dc1790281b1 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.487112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.487643 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.487919 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.488189 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] PATH_INFO: `/v3/domains` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.488822 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81386) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 12:07:02.489120 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.490728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.503858 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1447/2897] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Sep 12 12:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 12:07:02.509293 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.518894 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489417db-b8a0-4d77-9a12-ec5951a0509e None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.523593 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.524407 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.524689 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.524958 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.525589 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.527556 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.555598 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1451/2898] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:07:02.560055 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.570233 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43ebfb-4a6c-4c55-9563-d4d62abf065b admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.627464 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.628145 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.628363 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.628630 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.629294 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.631243 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.653591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1448/2899] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 12:07:02.657751 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.668752 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2801c8-e609-40b0-869e-463264dedd77 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.673350 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.673951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.674231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.674539 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.675185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.676779 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.688080 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1452/2900] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:07:02.692046 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.703610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654935c-3e69-4d7a-b72f-ccbdb9930e02 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.708214 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.708813 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.709093 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.709360 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.709999 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.711805 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f92ed4-d696-4a8f-a52b-84fa18757ae4 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.722400 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1449/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:07:02.727229 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.737588 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1c08-9507-465a-9fb8-322959461438 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.742185 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.743746 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.744093 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.744426 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] PATH_INFO: `/v3/projects/33a900721e33498e993f4dae51d9d76c/users/67c1854a4cde4be9b53c2a0b77b2441d/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.759758 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a900721e33498e993f4dae51d9d76c, user_id=67c1854a4cde4be9b53c2a0b77b2441d, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.762444 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.776053 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1453/2902] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:07:02 2022] PUT /identity/v3/projects/33a900721e33498e993f4dae51d9d76c/users/67c1854a4cde4be9b53c2a0b77b2441d/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:07:02.787406 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.787691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.788010 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.817907 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] MFA Rules not processed for user `67c1854a4cde4be9b53c2a0b77b2441d`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:02.819135 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:02.879675 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:02.887013 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1450/2903] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:02.891527 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:02.965406 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e0f67-801e-489b-b19f-97f1d97751f8 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:02.970064 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:02.970775 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:02.971182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:02.971462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:02.972107 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:02.974089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:02.997024 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:03.010176 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1454/2904] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:07:02 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:07:03.010741 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.022133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72df088-8135-42ec-bed1-07819ebf1734 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.084530 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.085166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.085449 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.086134 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.086389 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.088480 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.113087 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1451/2905] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:07:03 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:07:03.115157 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.127347 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32fb0f-a39c-4186-93bd-c8a259b63087 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.131996 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.132602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.132884 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.133786 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.134430 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.136060 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.146791 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1455/2906] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 12:07:03.151591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.161520 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2e438-3931-40f4-bb3a-422d5269db53 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.166882 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.167473 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.167754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.168028 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.168663 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.170247 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153ae46-6493-429c-adf1-aa244cf4bdd8 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.180430 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1452/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:07:03.184452 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.194149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca2c81-df92-4b65-a21e-c524f77f896d admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.198803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.199700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.199979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.200254 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] PATH_INFO: `/v3/projects/4286ec4c9dcc40a5bdee77f357a76ace/users/573b29868fff4948892a1bdd68617129/roles/7c78b3797aea4e3280d32765c45d23df` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.214937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4286ec4c9dcc40a5bdee77f357a76ace, user_id=573b29868fff4948892a1bdd68617129, role_id=7c78b3797aea4e3280d32765c45d23df)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.217586 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.231149 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1456/2908] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:07:03 2022] PUT /identity/v3/projects/4286ec4c9dcc40a5bdee77f357a76ace/users/573b29868fff4948892a1bdd68617129/roles/7c78b3797aea4e3280d32765c45d23df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:03.241148 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.241437 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.241714 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.273218 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] MFA Rules not processed for user `573b29868fff4948892a1bdd68617129`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:03.275263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:03.339570 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1453/2909] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:03.344007 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.410081 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae081a3-8c4a-4642-b2cd-1ad2f3d00895 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.415635 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.416257 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.416540 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.416819 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.417464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.419427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.452484 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1457/2910] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Sep 12 12:07:03 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:07:03.453649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.465680 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825698df-d674-4f33-8f35-15ee60ce9df5 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.525580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.526137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.526418 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.526739 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] PATH_INFO: `/v3/users` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.527372 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.529292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.540903 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:03.553133 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1454/2911] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Sep 12 12:07:03 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 12:07:03.557865 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.568134 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bbc85-f25e-43c9-841e-a5861aff50b1 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.573679 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.574651 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.575006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.575385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.576037 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.577666 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.589170 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1458/2912] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:07:03.593580 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.604070 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f43f1-8664-426e-a1b0-c6be871e68dd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.609293 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.610023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.610306 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.610626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] PATH_INFO: `/v3/projects/c5f7854ad4ac4907a840242883b881c0/users/fd6dab9349ea40b197320e496610a971/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.626943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f7854ad4ac4907a840242883b881c0, user_id=fd6dab9349ea40b197320e496610a971, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.629666 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.644801 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1455/2913] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:07:03 2022] PUT /identity/v3/projects/c5f7854ad4ac4907a840242883b881c0/users/fd6dab9349ea40b197320e496610a971/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:03.649630 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.659590 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b77a-415d-4d8d-bab4-f90210ca8258 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.717750 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.718363 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.718677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.718952 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.719595 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.721202 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1434f-85ba-4797-a180-c584c8d69d11 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.731811 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1459/2914] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Sep 12 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 12:07:03.736068 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.746955 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e893ef-8c75-48d7-a34d-a212c9cfea88 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.751457 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.752098 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.752379 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.752649 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] PATH_INFO: `/v3/projects/c5f7854ad4ac4907a840242883b881c0/users/fd6dab9349ea40b197320e496610a971/roles/ada491db6331406099f2a486b26a3cc4` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.754543 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f7854ad4ac4907a840242883b881c0, user_id=fd6dab9349ea40b197320e496610a971, role_id=ada491db6331406099f2a486b26a3cc4)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:03.757130 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:03.767629 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1456/2915] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Sep 12 12:07:03 2022] PUT /identity/v3/projects/c5f7854ad4ac4907a840242883b881c0/users/fd6dab9349ea40b197320e496610a971/roles/ada491db6331406099f2a486b26a3cc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:03.777708 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed68526-a285-4623-8816-5677519f430f None None] REQUEST_METHOD: `POST` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.777926 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed68526-a285-4623-8816-5677519f430f None None] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.778200 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed68526-a285-4623-8816-5677519f430f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:03.805808 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.auth.core [None req-bed68526-a285-4623-8816-5677519f430f None None] MFA Rules not processed for user `fd6dab9349ea40b197320e496610a971`. Rule list: `[]` (Enabled: `True`). {{(pid=81387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:03.807042 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bed68526-a285-4623-8816-5677519f430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:03.831700 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:03.878729 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1460/2916] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:03.890471 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:03.892200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:03.948842 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871d1f-bc66-4a30-835f-ece1b566e087 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:03.952961 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:03.953613 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:03.953895 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:03.954163 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:04.009597 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:04.011350 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:04.024223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1457/2917] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:07:05.580056 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed68526-a285-4623-8816-5677519f430f None None] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:05.581685 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-bed68526-a285-4623-8816-5677519f430f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:05.594622 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed68526-a285-4623-8816-5677519f430f None None] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:05.598182 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:05.598831 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:05.599116 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:05.599531 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:05.666789 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:05.668502 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:05.678282 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1461/2918] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:07:06.222179 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:06.231672 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0d09-6d2f-43d4-b5fc-c3c8daf4fc20 service cinder] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:06.235281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d11c4cfca65434c9108393d11c1f7ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:06.235878 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:06.236159 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:06.236429 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:06.245230 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:06.246678 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8b146-bd8d-49f0-8626-b39491275d0f service cinder] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:06.255938 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1458/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 12:07:10.176490 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:10.238937 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6957d-985b-4c46-a2a1-f040515b4e5f service cinder] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:10.243462 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:10.244085 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:10.244368 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:10.244643 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] PATH_INFO: `/v3/users/67c1854a4cde4be9b53c2a0b77b2441d` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:10.245765 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c1854a4cde4be9b53c2a0b77b2441d)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:10.248463 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:10.293975 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c1854a4cde4be9b53c2a0b77b2441d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.301212 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c1854a4cde4be9b53c2a0b77b2441d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.306494 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:10.315581 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c1854a4cde4be9b53c2a0b77b2441d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.316272 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c1854a4cde4be9b53c2a0b77b2441d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.323191 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c1854a4cde4be9b53c2a0b77b2441d'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.330323 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1462/2920] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:07:10 2022] DELETE /identity/v3/users/67c1854a4cde4be9b53c2a0b77b2441d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:10.334779 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:10.335292 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:10.335414 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:10.354150 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] MFA Rules not processed for user `d0a0e6dd3f1445518b190cea2b73f55b`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:07:10.355415 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:10.409664 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:10.419620 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1459/2921] 10.222.0.15 () {58 vars in 958 bytes} [Mon Sep 12 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:07:10.429565 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:10.430980 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:10.488356 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277717f9-040e-4db3-80bf-4596d977e85e admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:10.491963 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:10.492570 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:10.492850 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:10.493124 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:10.502086 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:10.503580 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:10.513120 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1463/2922] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:07:10.780057 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:10.781525 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:10.796201 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6216e-8d2a-489b-8fba-4a1f58b064ed None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:10.801856 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:10.802488 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:10.802795 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:10.803061 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] PATH_INFO: `/v3/projects/33a900721e33498e993f4dae51d9d76c` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:10.804250 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a900721e33498e993f4dae51d9d76c)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:10.806048 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:10.854177 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a900721e33498e993f4dae51d9d76c', 'request_id': 'req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.861487 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a900721e33498e993f4dae51d9d76c', 'request_id': 'req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.867137 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a900721e33498e993f4dae51d9d76c', 'request_id': 'req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.878469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] The token cache is being invalidate because project 33a900721e33498e993f4dae51d9d76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:07:10.878787 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.879422 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:10.880916 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1460/2923] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:07:10 2022] DELETE /identity/v3/projects/33a900721e33498e993f4dae51d9d76c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:10.884935 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:10.895198 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b4684-e4d5-4bcc-a40d-2cf853b10929 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:10.952169 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:10.952777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:10.953059 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:10.953332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] PATH_INFO: `/v3/users/573b29868fff4948892a1bdd68617129` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:10.954450 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573b29868fff4948892a1bdd68617129)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:10.956382 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:11.003566 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573b29868fff4948892a1bdd68617129'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.014358 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573b29868fff4948892a1bdd68617129'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.027538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573b29868fff4948892a1bdd68617129'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.028932 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573b29868fff4948892a1bdd68617129'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.036057 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573b29868fff4948892a1bdd68617129'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.038177 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:11.042337 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1464/2924] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:07:10 2022] DELETE /identity/v3/users/573b29868fff4948892a1bdd68617129 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:11.346431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:11.348005 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:11.363196 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450af69-8b1e-4c27-8c6c-1aa67d27f40d admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:11.431118 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:11.431754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:11.432044 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:11.432342 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] PATH_INFO: `/v3/projects/4286ec4c9dcc40a5bdee77f357a76ace` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:11.433479 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4286ec4c9dcc40a5bdee77f357a76ace)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:11.436166 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:11.487610 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4286ec4c9dcc40a5bdee77f357a76ace', 'request_id': 'req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.494646 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4286ec4c9dcc40a5bdee77f357a76ace', 'request_id': 'req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.502200 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4286ec4c9dcc40a5bdee77f357a76ace', 'request_id': 'req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.517652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] The token cache is being invalidate because project 4286ec4c9dcc40a5bdee77f357a76ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:07:11.520249 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.520940 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.523772 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1461/2925] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:07:11 2022] DELETE /identity/v3/projects/4286ec4c9dcc40a5bdee77f357a76ace => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:07:11.527104 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:11.529677 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:11.547774 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c453134-839c-4721-9f29-3b6b555fc018 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:11.607030 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:11.607665 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:11.608002 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:11.608217 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] PATH_INFO: `/v3/users/fd6dab9349ea40b197320e496610a971` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:11.609374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6dab9349ea40b197320e496610a971)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:11.611307 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:11.657785 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6dab9349ea40b197320e496610a971'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.665091 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6dab9349ea40b197320e496610a971'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.677456 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6dab9349ea40b197320e496610a971'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.678102 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6dab9349ea40b197320e496610a971'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.685882 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6dab9349ea40b197320e496610a971'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:11.692130 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1465/2926] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:07:11 2022] DELETE /identity/v3/users/fd6dab9349ea40b197320e496610a971 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:07:11.960149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:11.961575 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:11.974761 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f056e-ca1f-4ba1-b52b-ffe4cb0b63ca admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:11.987974 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:12.037339 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:12.037970 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:12.038245 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:12.038535 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] PATH_INFO: `/v3/projects/c5f7854ad4ac4907a840242883b881c0` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:12.039667 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f7854ad4ac4907a840242883b881c0)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:12.041626 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:12.093381 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f7854ad4ac4907a840242883b881c0', 'request_id': 'req-a78f3799-934b-4042-86f8-6a3ece6f1ebd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:12.100577 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f7854ad4ac4907a840242883b881c0', 'request_id': 'req-a78f3799-934b-4042-86f8-6a3ece6f1ebd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:12.107009 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f7854ad4ac4907a840242883b881c0', 'request_id': 'req-a78f3799-934b-4042-86f8-6a3ece6f1ebd'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:12.118281 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] The token cache is being invalidate because project c5f7854ad4ac4907a840242883b881c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:07:12.118611 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:12.119260 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:07:12.120754 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1462/2927] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:07:11 2022] DELETE /identity/v3/projects/c5f7854ad4ac4907a840242883b881c0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 12:07:19.292017 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:07:19.293497 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:19.352854 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c798ca9-31e1-4a93-be4c-5f7ab77b4b9b admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:07:19.386499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:19.406554 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:07:19.407172 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:07:19.407464 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:07:19.408070 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:07:19.409620 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:07:19.477615 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:07:19.479249 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:07:19.489767 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1466/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:07:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:07:22.115095 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:07:29.484704 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=34 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:08:17.287544 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:17.288939 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:17.301591 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f3799-934b-4042-86f8-6a3ece6f1ebd admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:17.305077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:17.305721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:17.306001 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:17.306270 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:17.308023 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:17.367554 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:17.368943 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:17.377957 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1463/2929] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:08:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:08:18.378956 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:18.387859 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d1294-bb10-46b6-bdfb-30bc0057ccff service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:18.392074 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:18.392757 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:18.393115 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:18.393460 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:18.395316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:18.480332 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:18.481809 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:18.494308 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1467/2930] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:08:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:08:19.676834 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:19.678423 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:19.690041 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09989cfc-aa1e-4829-871a-1b8b47c14fbd service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:19.693537 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:19.694128 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:19.694404 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:19.694705 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:19.696365 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:19.709469 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:19.710912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:19.721505 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1464/2931] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:08:23.387226 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:23.396583 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257c8e-b517-46e5-8321-bf89d85f6cf4 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:23.400374 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:23.400987 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:23.401265 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:23.401710 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:23.403979 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:23.473499 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:23.475218 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:23.488461 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1468/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:08:29.716721 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=34 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:08:30.270910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:30.272373 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:30.351652 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:30.353232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:30.373803 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e114628-ee4f-48d7-941f-093f17f41339 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:30.380316 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:30.380973 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:30.381625 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:30.381904 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] PATH_INFO: `/v3/users/72ff513b1b924f54818d7d3e38eb4a0b` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:30.387538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ff513b1b924f54818d7d3e38eb4a0b)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:30.389573 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:30.403701 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8b0e4-ce3e-410d-bcd2-c2cf975d4434 service neutron] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:30.410213 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324a072-29a7-4454-967f-341aab905133 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:30.411364 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a072-29a7-4454-967f-341aab905133 service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:30.411654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a072-29a7-4454-967f-341aab905133 service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:30.411931 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a072-29a7-4454-967f-341aab905133 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:30.413826 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-9324a072-29a7-4454-967f-341aab905133 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:30.431484 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324a072-29a7-4454-967f-341aab905133 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:30.433215 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324a072-29a7-4454-967f-341aab905133 service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:30.446859 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1465/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:08:30.453266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ff513b1b924f54818d7d3e38eb4a0b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.470602 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ff513b1b924f54818d7d3e38eb4a0b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.488310 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ff513b1b924f54818d7d3e38eb4a0b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.489158 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ff513b1b924f54818d7d3e38eb4a0b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.494731 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:08:30.497492 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ff513b1b924f54818d7d3e38eb4a0b'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.505672 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1469/2934] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:08:30 2022] DELETE /identity/v3/users/72ff513b1b924f54818d7d3e38eb4a0b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:08:30.680220 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324a072-29a7-4454-967f-341aab905133 service placement] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:30.692004 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324a072-29a7-4454-967f-341aab905133 service placement] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:30.748444 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:30.749073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:30.749263 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:30.749531 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] PATH_INFO: `/v3/projects/2862c2760b6548ae95e2ce59470ede1d` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:30.750756 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2862c2760b6548ae95e2ce59470ede1d)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:30.752648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:30.798691 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2862c2760b6548ae95e2ce59470ede1d', 'request_id': 'req-cb1e069c-f81b-4b13-ae06-97fd64e6f434'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.811016 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2862c2760b6548ae95e2ce59470ede1d', 'request_id': 'req-cb1e069c-f81b-4b13-ae06-97fd64e6f434'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.817112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2862c2760b6548ae95e2ce59470ede1d', 'request_id': 'req-cb1e069c-f81b-4b13-ae06-97fd64e6f434'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.829641 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] The token cache is being invalidate because project 2862c2760b6548ae95e2ce59470ede1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:08:30.829945 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.830654 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.832188 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1466/2935] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:08:30 2022] DELETE /identity/v3/projects/2862c2760b6548ae95e2ce59470ede1d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:08:30.836231 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:30.837636 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:30.851646 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf236c-81dd-44d0-86e7-905bf90ff604 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:30.910594 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:30.911227 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:30.911504 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:30.911777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] PATH_INFO: `/v3/users/f21ecb7ab0a64c75bca32a6d1ae58f60` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:30.912907 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21ecb7ab0a64c75bca32a6d1ae58f60)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:30.915232 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:30.963413 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21ecb7ab0a64c75bca32a6d1ae58f60'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.972306 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21ecb7ab0a64c75bca32a6d1ae58f60'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.985605 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21ecb7ab0a64c75bca32a6d1ae58f60'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.986277 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21ecb7ab0a64c75bca32a6d1ae58f60'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:30.993228 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21ecb7ab0a64c75bca32a6d1ae58f60'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.000206 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1470/2936] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:08:30 2022] DELETE /identity/v3/users/f21ecb7ab0a64c75bca32a6d1ae58f60 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:08:31.182836 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:08:31.184324 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:08:31.199730 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e069c-f81b-4b13-ae06-97fd64e6f434 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:08:31.219409 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:08:31.262729 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:08:31.263285 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:08:31.263563 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:08:31.263832 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] PATH_INFO: `/v3/projects/9c1b90516f3a43329901cb2fa0565c89` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:08:31.264934 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1b90516f3a43329901cb2fa0565c89)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:08:31.266867 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:08:31.312987 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1b90516f3a43329901cb2fa0565c89', 'request_id': 'req-7e0131e4-d72b-4ba3-a562-233ddf823bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.320888 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1b90516f3a43329901cb2fa0565c89', 'request_id': 'req-7e0131e4-d72b-4ba3-a562-233ddf823bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.326704 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1b90516f3a43329901cb2fa0565c89', 'request_id': 'req-7e0131e4-d72b-4ba3-a562-233ddf823bbf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.337760 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] The token cache is being invalidate because project 9c1b90516f3a43329901cb2fa0565c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81386) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:08:31.338047 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.338711 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-7e0131e4-d72b-4ba3-a562-233ddf823bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:08:31.340912 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1467/2937] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:08:31 2022] DELETE /identity/v3/projects/9c1b90516f3a43329901cb2fa0565c89 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:08:40.997044 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:08:41.334853 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:09:22.027777 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:22.029317 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:22.087448 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90546-e811-4c19-bc80-fa2bebdb6d60 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:22.139000 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca58b9ac9151436aad5a9546fdbe6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:22.139537 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:22.139816 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:22.140088 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:22.141815 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:22.151805 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:22.153220 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:22.161423 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1471/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:09:32.158564 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=46 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:09:39.521157 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] REQUEST_METHOD: `POST` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:39.521665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:39.521786 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:39.544873 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.auth.core [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] MFA Rules not processed for user `a17fa3464f414b2da364930144e85fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81386) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 12:09:39.546077 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:39.599073 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1468/2939] 10.222.0.15 () {58 vars in 967 bytes} [Mon Sep 12 12:09:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 12:09:39.638995 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:39.640415 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:39.708906 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c300-f22b-45e0-ae7e-8600185ea216 service neutron] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:39.714364 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091613b4778d4946b0059334bdb70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:39.714988 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] REQUEST_METHOD: `GET` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:39.715266 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:39.715538 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:39.723396 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:39.724951 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:39.733610 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1472/2940] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Sep 12 12:09:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 12:09:46.084648 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:46.087652 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:46.153031 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2abbd-52ef-42f9-b6af-427653f99149 None None] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:46.157547 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:46.158172 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:46.158452 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:46.158755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] PATH_INFO: `/v3/users/12b69e54d58048b99559368c0cc9d7cf` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:46.159881 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b69e54d58048b99559368c0cc9d7cf)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:46.161800 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:46.208393 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b69e54d58048b99559368c0cc9d7cf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.220112 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b69e54d58048b99559368c0cc9d7cf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.227835 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b69e54d58048b99559368c0cc9d7cf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.234728 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b69e54d58048b99559368c0cc9d7cf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.235356 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b69e54d58048b99559368c0cc9d7cf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.241665 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1469/2941] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:09:46 2022] DELETE /identity/v3/users/12b69e54d58048b99559368c0cc9d7cf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:09:46.415683 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:46.426543 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3119d8-752a-4179-a227-722aef11644b service swift] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:46.487006 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:46.487633 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:46.487912 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:46.488183 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] PATH_INFO: `/v3/projects/b3df59bc28534911b091999aa03aca2e` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:46.489289 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3df59bc28534911b091999aa03aca2e)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:46.491827 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:46.536449 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3df59bc28534911b091999aa03aca2e', 'request_id': 'req-359ffc33-c266-4f50-9b6a-e4a849b0c648'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.542029 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3df59bc28534911b091999aa03aca2e', 'request_id': 'req-359ffc33-c266-4f50-9b6a-e4a849b0c648'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.547878 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3df59bc28534911b091999aa03aca2e', 'request_id': 'req-359ffc33-c266-4f50-9b6a-e4a849b0c648'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.560427 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] The token cache is being invalidate because project b3df59bc28534911b091999aa03aca2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:09:46.560709 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.561345 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.563485 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1473/2942] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:09:46 2022] DELETE /identity/v3/projects/b3df59bc28534911b091999aa03aca2e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:09:46.567910 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:46.570054 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:46.583527 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80cb1d-261e-4b66-a040-64f71cb2d374 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:46.603149 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:09:46.643394 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:46.644022 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:46.644495 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:46.644947 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] PATH_INFO: `/v3/users/922b47f760de41d491eba3e43f0f2baf` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:46.646755 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922b47f760de41d491eba3e43f0f2baf)` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:46.649854 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:46.698183 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922b47f760de41d491eba3e43f0f2baf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.710259 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922b47f760de41d491eba3e43f0f2baf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.718747 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922b47f760de41d491eba3e43f0f2baf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.726223 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922b47f760de41d491eba3e43f0f2baf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.726890 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.notifications [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922b47f760de41d491eba3e43f0f2baf'} {{(pid=81386) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:46.732660 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1470/2943] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Sep 12 12:09:46 2022] DELETE /identity/v3/users/922b47f760de41d491eba3e43f0f2baf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:09:46.904425 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Authenticating user token {{(pid=81387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:09:46.905966 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.fernet_utils [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:09:46.924004 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ffc33-c266-4f50-9b6a-e4a849b0c648 admin admin] Validating token access rules against request {{(pid=81387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:09:46.942025 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:09:46.986739 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0a0e6dd3f1445518b190cea2b73f55b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1375001928324c1f9d2a4c3ccf24704e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:09:46.987385 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:09:46.987669 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:09:46.987947 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] PATH_INFO: `/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027` {{(pid=81387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:09:46.989089 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea63c75c6ac248d0b63c05f19fa2b027)` {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:09:46.992139 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] RBAC: Authorization granted {{(pid=81387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:09:47.041160 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea63c75c6ac248d0b63c05f19fa2b027', 'request_id': 'req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:47.046810 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea63c75c6ac248d0b63c05f19fa2b027', 'request_id': 'req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:47.052986 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea63c75c6ac248d0b63c05f19fa2b027', 'request_id': 'req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e'} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:47.065327 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] The token cache is being invalidate because project ea63c75c6ac248d0b63c05f19fa2b027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 12:09:47.065616 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:47.066234 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG keystone.notifications [None req-ad226542-a2a5-4d5e-a0d7-17d5bf42cd7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 12:09:47.067722 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: [pid: 81387|app: 0|req: 1474/2944] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Sep 12 12:09:46 2022] DELETE /identity/v3/projects/ea63c75c6ac248d0b63c05f19fa2b027 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 12:09:56.729303 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:09:57.062320 dv-cinder-853315-9-iscsi devstack@keystone.service[81387]: DEBUG dbcounter [-] [81387] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 12:11:21.269900 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Authenticating user token {{(pid=81386) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 12:11:21.271383 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:11:21.419104 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72163-0c21-4407-b4e6-9d3efa27a8b6 admin admin] Validating token access rules against request {{(pid=81386) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 12:11:21.479676 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27b96a89a0d140aeaeba8078a21b4cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77204d1f5d484097a21762f2a6bef64c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81386) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 12:11:21.480288 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] REQUEST_METHOD: `GET` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 12:11:21.480569 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 12:11:21.480840 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81386) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 12:11:21.482607 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.fernet_utils [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81386) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 12:11:21.493819 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 12:11:21.495431 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca1554-ca95-496f-8dc1-272bfe0ef6e3 service placement] RBAC: Authorization granted {{(pid=81386) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 12:11:21.505844 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: [pid: 81386|app: 0|req: 1471/2945] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Sep 12 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 12:11:31.501746 dv-cinder-853315-9-iscsi devstack@keystone.service[81386]: DEBUG dbcounter [-] [81386] Writing DB stats keystone:SELECT=89 {{(pid=81386) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}